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Preface 



This^report is based on a sfudy conducted by the Consortium of ProfessJonoi 
Associations for tbe Study of Special Teacher Improvement Programs (CONPASSj for the 
U.S. Office of Education. : * 

The Consortium was formed \f\ May, 1966, by the American Historical Association, 
the Association cf American Geographers, the Department of Audiovisual Instruction 
IfNEA), the^terrwtional Rea<;)ing A&sociatibn, and th6 Modern Language Associati9n of 
Americp. Invitations were later ext^nded-^o, and accepted by; the Americqn Economic 
Association, the American Industrial Arts Association, .and.fhe American Political Science 
Association, Four members at farge'provide liaison with the arts? and humanities, psychologica 
testj and measurement, educational' psychology, and teache^ education specialists. 

' The objectives of.CONf ASS are to provide a coordiroted-assessmenl' of the 
effectiveness and irnpocts of institutes and other specialiteacher-trairiing programs; to 
propose. means of improving such programs; and to provide 0 medium for dialogue amor>g 
the professional associations and leading scholar? of the several subject content disciplines 
and fields represented on its Bogrd. In the past,* the Consortium has conducted studies of 
summer institutes in individual discFplines; it is presently^ sponsoring a'n extensive study of ^ 
the impact of summer institutes in* four. disciplines upon participants in the ii^stitutes. • 

The "presenAtudy of the Experienced Teacher Fellowship Program was contracted by 
CONPASS to Ctark University, to be conducted under the supervision*of the ConsoFtium 
Board-. The research was inifiated Professors Crocjcett and Bentley, Professor Laird 
participated in the analysis of the results and^in the writing of the report. The res^ch 
staff spent four days m a writing conference in July, 4967, with Drs. John Thompson, 
Sou! Cohen, William Engbretson, Richard Longaker,^nd Mr. John Cogan; gt this copference, 
the^results were studied in detail and the outline of the present report Was formulated. 
Preliminary drafts 'of the report^ were examined by th^ members pf the writing coyference.and 
by the Executive Committee cf the Consortium; the final versioh o? th6 report has beriefitted 
extensively from their comments. , 

This edition of the replort ftas been preparecj in October, 1967, for advance circulcitio 
A later," bound edition^will be issued by tke Consortium Office. 
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I. Intiioducticm 



The Experienced Teacher Fellowship f?rogram'is a unique. and irnaginative venture: 
Its ultimate objective, and that of two other, closely reloted'programs, S*s to improve 
the quoflty of educptipn in .thq nation's elementary ar^d secorxJary schools. The three 
programs pursue this objective in two ways: b^ assisting selected, potentially influential 
teachers to pursue full-time graduate education in spfeciaMy planned courses of sfudyy 
and by fostering and strengtbenin^an increased concern fouthe tralnmg of teachers. The 
Experienced Teacher Fellowship Program sponsdr$ ipeciaf programs that provide financial - 
support for graduate studies to teachers witfj' field experience, A second program. Vie ^ 
Prospective Teacher Fellowship Program, supfjorts simi lar kinds of programs for individuals 
who have no teaching experience but who expect tO-become elementary or secondary 
school teachers.' The'^third ^program, the Institutionol Assistance Grant Program, awards 
financial grants to strengthen the graduate programs for^teacher preparation in 
insti-tutions that have already been awarded either an Experienced or' a Prospective Teacher 
FeMowship Program. The present report summarizes a preliminary study of the Experienced 
Teacher Fellowship program , ^ 

A. The Development of the Experienced Teacher Fellowshij^rogram . 

The history of the Ex TFP belies the generalization that governrnental programs i 
develop sipwfy. It was authorized under Title V, Part C of the Higher Education Act of 
1965. G\iide lines for the program were distributed in two letters, dated December 27, 1965,- 
and Janyary 10, 196^, The deadline ;for mailing completed proposals was January 20; a 
pan^l of. consultants read, gvaluoted, and rated the proposals in the period>from January 24 
to 26; and the announcement of awqrds waa mode in.Feljruary, barely two months after the . * , 
first guid^eline was sent out. The first students began their study in June, 1966. 

< - * 

De^spite the speed with which the program was mounted, alryioSt J ,000 proposals 
were, Submitted for the academic year 1966-67. Fifty of these proposals were»funded, 
eriabling just over a thousand ^xperiented teachers, from all parts of the country and representing 
diverse disciplines, Jo spend a year (in a few programs, two years) in full-time graduate shjdy. 

In its unde^rlyjng assumptions, the conception of the f^^f^P was broad and inclusive. , 
In the guidelines, no limits were suggested as to the, range d^^^^uB^ect matter that would 
be supported; no premium was placed on either innovation or traclitionalism In educational 
procedures, and t^iei^e was no attempt to specify in detail the structure that the graduate 
programs should adopt. - There was, however, the assumption thcrt- graduatfe education js 
' most effective when the courses a student tak^s are related to one another in a meaningful • 
fashion. The guidelines for E^TFP proposals incorporcited this assumption by setting three 
restrictiqns on authors of propolis: * * ^ 

First, evidence was required of more than perfunctory cooperation between subject- 
matter and tea'cher-education speqkilists. All proposals were required to demonstrate that 
a suitab^le' faculty could be arranged for, composed of members of "teocher education" and 



"non-teache/ ^qucotion"'departnients Further, both the chairman X)r dea^^ for th)fe si/bsj-antive . 
aspec^of the prcfgrcuh and. the cihalrman or d^an for teacher education^ wer^^ required to sign- ^ 
the 'propo^sa I before Tt ^as submitted . , 

Second, institutions *were required tb adopt an en bloc pr6cedure,, by "Sfesigning 
a program for the entire group of fifteen to thirty fellows/ rather, fhcm leaving t^*ind1vidual S ^ 
fellows "to tl^e^mercy of the ^catalogue ^sjcafeterid- like-offerings, often unsutted to the* • • 
needs of experienced personnel . " Th6 en bloc mode of organization was also t6 jprovide 
greater visibility of the'progranrl on fKe campus as welhas increased opportuni^ f^jf^iellows 
to profit from interaction with their peers^and from,-formal instr,uction .by their prof^yors. 

V ^ , * ^ 

/ Thirdy the' guidelines encouraged cooperation befween ^he ins^itutIon«of. higher 
edcrcation and the local school, district o^ syste'm . This was"f6stered/1ri part by the' reqc 
ment that fellows be seiected jointly by their home educational system and by the coHeae 
or university concerned, *Schoo(j' administrators were required to recommend applicants, 
and applicants were expected to return to the school systems from which they 9ame . -^n ^\ 
additi^on, in Order to confront the realities of teaching in schools, cooperati.on was encourbged 
between colleg'esand local school systems, t^^rSvide a meaningful practicum experience * 
for the participants. ; * • . ^ •.. - 

The fiftycprograms tha/were funded we/e held in forty-seven different coHeges and 
universities. Programs were ycbnducted in 17'different disciplines, ranging from general fields 
of education (elementary education, teaching the disadvantaged, and counseling and gfuidance) 
'through the traditional liberal arts disciplines, and including specialised areas such as' %^ 
health education, the school library, <5nd educational naedla . The fellows were drawn from 
every part of the country and from schools which, served every economic* Jevel; their educational 
assignments raraged from preschool to hRgh schbpi . \ > ^ ^ 

B. Evaluation of the Experience^d Teacher Fellowship Program ' " ^ ^ 

1*, Three Projected EvaluqU ori Studies • ' ' 

Just as the ExTFP was planned and- insti"tuted witin considerable spVe'd, so, also, were the 
procedures for studying^the program V effectiveness . Barely thre^ months elapsed between , 
the-formation of a research team-and conTipletion^of data collectipn for the present report, 
Durir^ that tirrie, a pidn has taken fofm which foresees a serle/of three related investigations 
of the effectiveness of the ExTFP: a questionnaire study of responses to the first year's 
program; 'during the second year, 'afield investigation of the pperation'5f the ExTFP in three 
different institutions; and, in the third year, another study of the entire set of institutions 
then involved in tfie ExTFP. Each successive investigation will build on the results' obtained 
by^ those preceding . i ' ^ • ^ ^ 

C The first of these ^tu^dies"*, based on questionnaires and visits by teams to^selected 
programs, will be described at length below, Tl^e intensive^ilot study of three individual 
programs will be carried out during-the acadernic^year 1967-68; it will involve repeated 
interviews with participants and faculty in each institution arvd the periodic administration 



of questionnaires ond other tests,. The exter^sive body of information thot will be obtoined 
wil I, p^niiit on acquaintonce in depfch with th^ operotion of these thHre^ institytiopj, moking 
impossible \o identify focrfors thot appear to oiscount for the effects the programs hove upon 
the fellows, the focU^ty, ond the institution. An importont ospect of this second study will 
be intervfews wil^h the fellows of/er they 'hove^ returned to their "home sch^ls in 196B-69.* 
T|ie thicc^ investigotjon, Wbe initfoted during the\ocademic 'ygar\19^8'"69, wil^ be on 
ej^ensive study of oH the Experienced Teocher Fellowship Prbgrams then in Operotion, 
using self-rjeport meosures, interviews", ond observotions, ol I'deVeloped out of eorlier 
research experiences, 

' 2 . The Proc edures Used in the J^resent lnvesti^a\ipn 

' The present report rests upon two kini^s of doto: responses to questionrftires * . 
thot were odministered to the individuals who? were'octuollys.irlvplved in the*program, .ond / 
report^y teoms of evoluotors who visited 31 of the 50 programs. ^ * * 

Jn ■ ' ■ ' ' . : : \ ■ , . ' i 

^ Four questionnoires were constructed for ChdministrationVo thos^ involred in the 
programs. Eoch questionnoire borrowed heovily from fhQ3e used\n eorlier stucikes of , 
summer institutes. One questionnaire, contcfining some 60 <Jifferm%t-*tems, wos. odministred 
t6 'the, fie Hows ot the institutioris t^y attended, ^under conditions which assured dnoq)^mity. 
Completed questionnoires were obtoined from 940 of the 1,004 fellows, represe'ntjng 49 of 
the 50 'fnstitutions . ^ ^ ' '^i ^ 

The director of eoch instij-ution wos dsked to. supply the narnes of the full-time 
ond parf- time stoff 6f his progrom, A cop/ of o second questionrthTre^ ooout equol in# 



length to the student questionnaire), wos then rpoiled to every full-t^rne faculty member 
ond to five random ly-selecte'Jkport-lirae foculty members on^epch compus .\ Completed - 
questionnoires w^e obtained from 18/ faculty members, in 47 differ|nt instUotions. 



, . A third questionnaire, sent tb thejjl'rector at eath institution, wos i<^emico| to 
the foculty questionnaire except thot it (fbntoined on odditlonal set of t^n items\concernihg 
the odministrotioji of the4)rogrom. Of the 50 directors, 45 returned tb«e questionnoires 
in time for*onolysis in the present report. 

The fourth quesH^nnoire, intended to ossess the Impoct of. the progrom upon the 
existing teocher-educotion procedures ot the institutions, wos serrt to the dir^tor oF teocher 
education on edth compus. Response to this questionnaire was $potty^ for^this reoson, these^ 
replies wlfl not he discussed in detoll in this report. 



' Quest ionrKij res were ncf/ receiyed from the program in Social Studies ot theNjniversI 
ofMinnesoto. . * 



^ ' *Thfe evblua/ioTi teams, which visited 31 of the SCT programs, normally consisted of 
three persons':' a specialist \n f\)e subject matter of^tJne institute, a spiecialis»t in teacher 
educatiqf , and o teacheit experrenced in the^ relaxant subject matter/] Some ^ mefnbgrs 
of these evaluation teams fnet in early April with the research teanrt, mem'lp^rs of COKlPASS, 
'representatives from the Office of ;Educgt(On for q discussion of the evhiuation rating scale 
bnd oF the procedure that was to be foNowed in. the evaluation vrsit . .They thfen spent two 
days on the campuses to w^iph they w^re ossigned, rrjeeting with foculty, students, artd 
administrtators/ visiting classes, and reviewing the general operation of the programs. 
Subsequently, each tearii member individually completed a Visitors Evaluation form-, 
contoining 24 different i±effts . For each, item, )he evoluator rated the program on d 7-poir^ 
scale, ^nd tHen wa§ asked to provide a written analysis of that aspejbf of the program's 
operation in explanation of his rating. In addition to the i ndiv id uav^re ports, the feam 
meThbers submitted a combined evaluation on each item of the evaluation form;, this last 
report represented the consensus of .all the team members. ' ' • ' % 



r 



'Because* of ^iffid^ulties in schedi>Jing members Of evaluation teaml7 three institutions 
were vIsUed by teams of only^ two members; at two ihstitutions, the tecvp contained four 
memberi^. ^ * » ' ; 



^ \ > • II. .An Overview of ^he' l^Ssu Its ' * / ^ ^ 

. ..... * - # • • . ' ^ 

In |he following -chapters ajcjfetaired report will be made of the results of,^ tbl^ study ; 
Jhe' pt/rpose af .the pr^sernt cWTpter li \p, point out the highlights df these results. , , ^ 

' ' \' * - , 

'* .For tfe^ mast part /fellows whc^J^ok part in the Pr>Dgram were, relatively yogng . 

' teachers, but experfenced^ornesj^ Tf\eu|^ibility and/rnoti'votiQn . 

\h\g\r; according Wpr-oJvpit^Irf <^r6i^^ the f^Mow^' weYe .'at -lieast equal " 

if not su|3erior7-|h quality xir>cP'rfi4ti vat ipn to tKe regular gra'Axite students at'the institutions^ 
concerned . Th<^ educational att.ainmen>6 and exp^rien^e of faculty members sugi^ests^that • ' 
•fhe average teacher irith'fe Prograro^, also, was more than qdequdtely^qualified . Thus, the-* - 
great majority of the fifty programs possessed-.thf +wopry'ncipal qualifications for an eff*ectTve * 
academic program:, an able* highly motivated jtudent body and*a capable, concerned faculty. 

The extent to whicMhe Program's poteliti a I effectiveness was/eoHzed and th^ • 
general correlate^ of- effectiveness mdy^ conveniently be summarized by six broad geneSra lizations; 
for the s*peciTic results on ;^h*tch^these generalizations.are based, the reader shoul^ consuli^ 
the body of the r^pbrt. ^ ' ' " ' ^ "* 
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, ♦ 1 The reaction to the Prof^m by program directors, faculty ^riembers/ fellows.^ 
and evaTucfrion teams was ov'erwhet)THryy Favorable. ^' Z •I / ' 

As a genewijrule, t+ie ^jjtent of a source^ enthusiasm about th'e Rrogram varied v 
with fhgt source's degree of professional investment if^ii: directors responses wer-e usually /nore 
fa^rajble ^n those by fatuity members, faculty members were more favbrerble than fellows, 
and fellows more favorablethan evaluators*.* But^this generbl rule held within a context 
of over-alt fa vorableness toward the prograrp. Specific Evidence of the^^idespread ' 
. approval that. was g||perated may be f^nd throughout the; results. The^xtent of tKis . ' 
approval may be illustrated by the^-fact that. 32% of the fellows reported that their pw« 
pr9grarn was either usOaJly stimulating ^nd interestin^'qr stimulating qnd interestirig throughout*. 
Responses by faculty members and pro-am directors to the identfcal item were evert more ' 
.favorable. Similarly, thernajority of,>^espondents in each of tWe four roles — directors, 
.faculty members, fellqws, arfd eva^Tuatch — reported that the ^rograTn had clearly^met the ^ 
educational needs of th^'fellows^ Even\the fe^w eyaluatiC^n teams which were^harpJy critical * 
of an indiviidual program took cpre >o comment favorably .on the over-all concept of the ExTFP. ^ 

■ ' - . ' X . • . . ^ 

^here was, of course, a considerable variation among nastitutions ^'n the evaluations 
that werey^^celved: Jome progxami^were given extr^mily hIghjpaTings, a few received 
relatively lovy evaluations It must be stressed again, however,' that thi^ ,varidtibn tocJk 
place around-an average value that was V^^IV favorable, indeed. ^. ' 
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2 . fRere- wos o high degree of solTdority qyd morole om of^feliows in the averoge ^ 
progtttm; the level of solidarity ond morale correlated p^t^yely with judgments of effectiveness, 

Agointh^re wtis considerabk variation' i<i the level j^r morale crffd solidarity that ' - 
* chorocterizeTlne different-institutions; nevertheless, , the average program recei>/,ed high 

roHn^s on morale a>nd ^olidgrity from directors; faculty membei-s, fellows, aruJ eva I ud tors alike . 

Beyond thls,^- there Wer^' ci^fis intent positive correlations between eit^mates of solidarity and 
'^morale arnl judgmehts of the'' effectiveness of individual programs? programs where morale 

and solidarity were;high also received high ratings on effectiveness and parti cfpant-sotisfactlon; 

when mbrole and Isolidarlty vyere low, so wete ratingT^f satisfacHoh qnd* effectiveness. 



.... v. . 

^ * 3, The amount of work assigned was heavy, and inversely related to e^f^c^Weness ' * 
« and dfiti^faction, • ♦ . / . 

t - QF the 31 evaluation teami, none ^aid the felbws* work loai 

^21 sqid'if wos In some degree' too heavy;^oT 940 fellows, onfy 6 sdi^jHe loa)|pHp>9Plht 

and 440 soTd it was "too heavy to alloy/ completion of assignments and inj^pGnqen? workr, 
Qf particular ioterest^Wbs the invferse relationship betwe^en feJIows* and evaluatdrs' 
judgments on this questiontlnd/the various, measures of satisTactrbn and effectiveness: 
institutions where the amount of work required was judged to be inordjnatel^high v/ere 
. consistently ranked os relatively .ineffective , If should Be noted that this relationship), held 
' for the absolute amount of work that was required, not for the amount of compe/i^veness that was 
fostered between fellows. In the average program, fellows reported,a fairly l^rge am9unt 
ot ccttnpetition with one another;, however, these latter ratings did not relate consistently 
either, to judgments of the over-all^ work load, or to ratings of effectiveness and satisfaction . 

4. ^Respondents indifferent rules' disagreed as to whetfrer the^rograms built on the 
extensive bac kg round s^of felloWs;jjudgments By fellows and evoluators on this question . ~ , 
I correkite d positiyejy with measures of effectiveness, \ 4 * 

^Almost all of the program directors and* a large grrajority of the fqculty^e ported 
that?he currijculum at their JnsTitutjon u-tilized and bwlf*upon the experience of the .fellows; 
most of the eva luotior^ teams indicated the reverse; ofiswers by the fe I Ico^- to- this, question weje 
intermediate, but more sirflilar to the fatuity's than to^e evoluators'. Despite their 
' disagreement in the level of which they felt the fallows' experience was utilized, evaluation 
^teams and fellows qgreed in thetr rankings of institutions on their achievement of this goal . 
furthermore, those^ programs whlch,\accordrng to fellows^and evoluators, managed somehow^ • 
toljuilrfupon the fellows' experien<SB received more favorable ratings on program effectiveness 
than those which did not. ^ 



' 5. Respdndents in different roles disagreed as to th^ extent of cooperation omQr>g 
programs and the amount of iranQvation in the programs; departmental cooperation, but 
not iftnovdtion wa& gory lated with program effectiveness, » \ ^ 

ForJ-he mo»part, "Sirector^ and'f^acOlty members, involVed in the Program refported that 
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^cooperatifeiramong different departments was good, thot th^ Program hod considerable' " 
efffects on teacher-education procedures^ aod'that |t contributed to the plans for development 
of^the de'partjment and institution conderned. Evaluation^ teams did not mal^e suc\^ favorably 
judgments^ Although* in some institutions eyaluators said. that the Program had aflFi^ted 
interdejjortmental cooperation, teacher traming,\or dejjlj^irtmental development) in as many 
other TnsHtutions evaluators.felt that it hod little effee^on-such policies. It s^enr^Jikely- ^ 
that the evaluatOrs' jch;)grnents were somewhere loser to'Tealit/ than those of thl'wector^^ 
6nd facJty; itj^ within t^e few months of the Program's pperation it had effecte^lew, if any, 

^lllpally sob^tontiaf changes inlhe strut^pr^e of most of the host institutions.. It is ^<fth 
noting, however, that ther^*was a positive correlation between' [ydgmerxts of progRinl 
effecti\?Wness on the on^ hartd and, on the other, reports -by evaluators'and by felTows of 
'effectfve** interdepartmentaf'cooperation. , ^ ' ^ 

^ As with lodgments of effects upon ho$r|nstItutfons, prograr?r directors and faculty , 
members were much more lil^ely than evaluatoii to report that th^ir Programs contained , 
' imaginative innova4ions. Agqin it is likely that the evaluators' jug||ments were more ^ 
objective than- those by the pfher two groups, and that a* rrvany programs introduced few 
educational innovations as introduced many.' It is noteworth that evaluator estimates of 
the extent of innovations was uncorrected with judgments of program-effectiveness. 

i , As adequate a summary, as pny of the effects of the Program upon the host instifjition* 
can be given by quoting from an interim report on th*is project, written in May, 1967: 

The Programs appear to have been least effective in ov^rcomiqp 
the traditional patterns of organisation i(^oolleges and universities. ; 
Thus, -the most common ppmpfaints ^y evaluatron teams^ dealt with 
the simUarTty of these programs to traditional undergraduate and v 
graduate ^education, the imposition of 6 common body of required 
courses upon al^gortitipants, the fpi lure to adjust the curriculum to 

• the needs of irrdividuql students, the absence* of true collaboration ^ 
between different departments of the same institution, or the unconcern 
of the stQ^f for the response tp the J^rograrh of the f eJtjows ix% q group^j 
All of these complaints are 90mmonly voiced throughout higher ecfucatiori^^^pr*V 
in America; they are'not unique to th^Experienced Teachers Feillowsh?p ^ 

• ^ Program. It is significant that a consicferbble number of institutions 
: ^ were adjudged to provide for their fellows' an unusual and rewarding *^ 

educational experience, some by folloN^ing traditional educational 
* pattern^ other by breaking wfth tradition and estahlishlVig novel and 

• ^ exciting educational proQedures. • 

6* The program director has an extr emely important role in deterrnining the . . 

e^ectiveness of indivlckiol programs. 

A partial enumeration of the functions that a programed ireptor perform^yields a list 
of impressive length. He should be directly involved in deciding upon the course content anid 
the mode of organization of the program, he mij(st^make sure that the formal courses and ' 
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sifiipplemental activities are coordinated, must ar/ange for the presence of whatever 
i^^catiolial rwdt^iat^ are r^^uired,. must encouroge informal exchanges among Inflows and. 
between fellowsrx±rid)staff, must' try to* mediate in disputes that may develop among 
participants, imust ascertain the fallows' and ^he faculty's views and criticisms of the" program 
,and its effectiveness, must decide , whether changes jn procedures or content are required, and, 
when the decision is affirmative, must determine what changes to make i^n the program^ 
and how to make them*. In a program whose succejss relies in good part upon tf;ie ^ ^ 
establishment of high esprit djs corps among participants and upon the group's performance - 
en jploc^ .the fulfillment of these furxrfc^s can be critically important. There are doubtless 
some (#ogfams which^run smoothly from begirining to enid never requiririg the mediq^ing 
influerlce of a skillfij- administrator. In the typical institution, however, at some time 
during the year crises ^rise, interests conf^1C^, difficulties occur which require effective 
'odministr.aifve action. At such times it is essential that the program director possess the 
ability ^'^e timej and especially, the institutlorxal power to resporxJ effectively to the^'dfemands 
of the sitiwt^n. • ^ ^ • • ^ • ^ 
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The PopulaHon anSS the Programs 



i 



- /We have already remarked fttat the ExTFP embraced a widfe variety of offerings. 
(Vograms varied not 6nly in the type of ^'nstMutions thatyere invglwd, in the gepgrophic v 
region wh'e^ the institutidhs were located; in the .characteristics of fellows they 'enrolled^ 
and in the subject mpgTsr /hat they offered; beypn^.thls, they^^jffer^ remarkably in the 

tdogogical strategy that the.y adopted > s^me were innovative, others traditional; some' 
ove for competitiveness among felloWs, Other fought a 'non-^|^ilpWftitive_atnioJf)here; 
some established informal ^^ationships bet<«<^5en'fel lows and faculty, others mairttalned 
forrpal relationships. Thetfii^rams and tW^^jndivrduals involved in them differed in these 
general ways and in alliother ways. in wHTbn'peo*pJe differ. Our purpose in tbfs.section' 
of the report is to summarize s^^ of<he characteristics of the fellows, the faculty, and 
the programs* In succeeding^ecttons, w^ sliall discuss reactions to these progratns^ aqd 
shall look for variables that correlated*wi|h their effectiveness. It may *sometim&j apP^^r 
that the differences among programs are obscured ir^ the course of ihis anafysis, that diversity 
is redo^ed'to uniformity. If so, the redder shodid bear in r^'d that we are seeking for 
^whatever underlying con^tancy^ there may b^ benWath the remarkcj^ble surface diversity. ^ 



A. 
\ 



/ 

Characteristics of the Participants 
1 . Personal Characteristics 



ot entfreiy * 
joubf a higher, 
s relatively young 



^ In certair\ of^their personaJ characteristics the group of fellows wOs 
representative of teachers as a whole. Men mode up 51% of the grobp, no 
proportion^ than obtains amdng teochiers in general . In addition, the group 
*with 79%. being youDger than fW) ajjd 28% younger than 30. Despite fheir re^Hve youth, the 
participants were not inexperfenced in teaching. Mjnety-two*percent repbrtcd three or 
more years of experiQi^:e in eduction; 5?% had six or more years of experierlce; however, 
only 24% hod ten or more yeors* of experience . The .participants' experience kpanne^ all 
levels of elementary and secondary education: 32% had been principally involved' at the 
' higji school tevel, 2*1% at the junior high schpol level, 45% of the elementar)?^level, and 
'2.6% In preschool or kindergcfrtett^teaching . ' - • 

^ Apparently, the fellows'^eonsiderpble experience and trofning had nofbeen primari|jy^ 
in the subject matter areas of their respective programs, for sixty-one percent fcad worked as 
"specialists" in their preas'for le^s-^Tian 3 years, while only 28^0. reported tajciiib as many 
as 30 semester hours ^f undergraduate credrt in their specialty -J- the presumed pquiv^^lent^ 
of an undergraduate major. Seventy percent had taken^^ewer^an 10 hours of 
credit in their special area', and a third reported no graduate cburses at all in that area. 
Two thirds had never 'attended on NQ^A summer institute qr similar training pro jiii^r^nd 
only a tenth had attended fifiore than^one such irogram. 



• . • ' 

* * * 

It is noteworthy that only 3% of the participants had held administrative jobs ^' 
when they enterec^the progrom. Of those not in pdmi^i§trative positionVv'55% said they 
' "probably" or "definitely" would' not go* into full-time educatiorKil administration, while 
only 6% said-they ^Jefinftely expected to do so. Upon completion of fhe program, these 
fellows will doubtless be in a favored position on the promotiorwl ladder of their home 
school 'systems;* their apparent relu'gtance to move into administrative positions bespeaks a 
strong .commitment to classroom teaching. It will be. important to examine, in future years, 
the actual career patterns of these fellows. * • * 

2. Characteristics of FelloNvs'- Horpe Communities and Schoals 

Fellows we^re distributed accprdirig to the size of the communities they came from 
in numbers roughly proportiorKil to the distribution in the population as a whole. Thirty- 
three percent wer^ from^ conrwrfunities with 'less than 2,500 residents, 40% from towns or * 
cities with populations between 2,500 and 100,000, .16% from cUies between IOO7OOO ^ 
and 500,000 population, and 19% from cities' of ovej 500,000. Only H% of the fellows', 
identified their school system as^bein^ in a suburb* or satellite city^ 

At least s4me participants came from each^part of the country. The Western and 
, Midwestern stafes^were somewhat over^-represented, with 24% qnd 33%, respectively,"^ 
..of all participants; 23% of thetfellows were^from the North Atlantic states, about -t+re ^ 
some pYoportion as in the population at large;»the Southeastern, Sputh Centro^^ and 
Southwesrtern stdtes were somewhat under-represented, comprising only 1 9V0 of the totol. 

f As t6 school enrollment, there were fellpws from schools wifh fevyer than 200 sti>dents, 
others fVom, schools with over 2,000, and still others in every category intermediate between 
'tfiese ^firemes . t * • 

Most^ommonly, fellow^ reported that their students came from families of middle 
income. Families with, low but steady ipcome were reported next most frequently, anc| either.^ 
wealthy ^families or those in poverty werp reportedly o small minority of the dlient^le 6f most"> 
fellows* schools. Sixty-eight percent reported that the pupijs \n their home schools were 
"all or mdstly white\"; the rempirvder reported that their pupils were predominantly Sponish- 
speaking, Indian, Negro, or a combination of two or more ethnic groups. Only about 15% .of 
the school-age children in Anperica are nonwhite; therefore, it appears that the proportion 
of teachers In the ExTFP who came from classrooms with substantial numbers of nbnwftte 
children was somewhat greater than in the natlorfxjs a whole. SiVice five of the 50 programs ^ 
were for teachers of the disaclvantag^, such an outcome is not surprising. . ' ' 

• 3.. Ability and Interests of Fellows • ' ' 

, No' information is available concerning the fellowsi^erformonce on standardized . * 
^ tests of ability. 'However, there were iVems on each of the quest ionrxj ires which requested 
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/^fellows, faculty, and direcj-ors fo estrmote/the fellows' ability. By all three sets of judgments, 
- the fellow$ came off extremely well. Thus, 83% of the fellows said^hat the participants' 
ability'wos above average and 95% reported that fellows seemed gendine-ly interested irr' 
the subject matter, " /\ ' - ^ > ^ - 

The faculty and directors werp osked to compare the ability of the ExTFP fellows 
with that of their institution's regular graduate students^ The results of theie comparisons; ■ 
are rflimarlzed in Table ] , It is clear'that the feliows were viewed very favorably by both 
sources, with program directors being consistently more favorable than the faculty. Note that 
morfe than half the faculty and directors repprted that fellows were more irdustrious, mprd^ 
serious, and had greqter initiative than their regular graduate students; nearly as much 
preference was given to fellows oyer gradudte sfudents in their commitment to the discipline * 
and their ability to communicate. The facujty thought that the two groups were about 
equal in intellectual ability, while directors favorfed the fellows; siriiilarly, the facCi|ty 
* rpted graduate students somewhat hTgher than 'fellows irr. knowledge of the discipline, while 
^ directors'^tings were the reverse: Considering that graduate students constitute a very 
select '^rt)i;p for comparison, these 'results provide an extremely favorable picture of the 
^ fellows*^ capacities, ' * ^ . 

Table ^ . Comparisons of Fellows with Typical Graduate Students-iMade by Faculty and 
. ' l^rogcom Directors * • ^ . • 



Variable 


Source of t 
Rating - ' ' . 


^ ^ ■ . Percent* Who Rated JParticip; 


ants 


verier 


Equa 1 ' 


% Worse 


InfellectuaJ 
ability 


Faculty / 


24 . 


46 


. 28 


DJiqctprs 

^* — - 


- i9 


56 

4 


13 


-7 — — cr: 

Industriousness 


1 

Faculty 


60 . 


'33 


2 - 


Directors ' 


65 


V. 31. 


0 


Seriousness 
- ^ 


Faculty 


. '66 ^ 


r 

. 27 


3 


Dire^tdrs. ' \ 


' 67 


29 


0 


Commitment 
to discipline 

i 


• 

FoGulty 


46 ' ' \^ 


34 


14 


-Ofr€cfors 


67- , . 


20 

• 


9 


Knowledge^ 
of dUciplirie 


* Faculty 


25 ■ ' 


38 




Directors 


■ 34 


■ 38 • 




Ability.t<S^ 
communicate 


' Facuky 


- 40 . 


' ' 45 . ' ' 


13 

, % 


Directors 


• 


44 j 


9 


Initiative* 


Faculty 


53 *- 


40 


4 


Directors ' 


■ 64 ^ 


— 


4 

m ' ' 



\ 



^ Since non- respondents are not included in th 
nottotalJOO 



is table, the percentages in each row do 



It IS of interest, as weH, fo examine what the felloes considered to be their greatest 
. problem, before enrolling tJi the ExTFP. They reported as follovs^s: , 

-•'.>_ ^ ■ ■ ■ \ • y 

2J3%^ Using effective teaching . methods- 

20%^ Knowledge of The subject matter * , 

' ^16% MotivaWng students ^ " » ' / 

uetefm]nih§- what is most importdpt^ teach * ' • • • • 

' Handling students oP low ability ^ 

6% Knowledge of appropriate mdtq^rials^ / ' \ <^ 

,3% EnfiQ0»qging and stimulating gifted students ' «^ t ' ^ 

One derives from the^e 'data .a picture of q young, ^nArgetic, serious, industrious 
groufuof tepchers, with considerable experience and jji stropq commitment to their w9rk., 
T.be communities that the fell'ows'were^drawh from seern to be approximately representative 
pf the nation as a whole except^ pei4iaps, that the South was somewhat under-represented 
'and that sihools wi^th subit6ntial rwin^ersof npnwhi,trf studferits, were, somewhat Overf-representecJ 
The fact tfxit felloWs weie relati.yety-.untralned'jn the^spedial^j^ed subject mcftter of their 
\ pogroms, combined W\m their inte llectual a&iUt^ ar^fl thei^r siVcerrty'df^/purpose; suggests", 
that th^y were especially liVeijf to tenefJi.frpm fh%ir.gVaduaHe work; , w . 

^ B Charocteristies ofth? Facul>y ' \. ' ' ' ^ ' ( " 

. '"Thie educational and professronal bacJ<groypd -^faculty; memj^ers ^'^ihejjfn^P ^. 
^'wos impressive. ••S«eveilty-three percer^f h^d e\\h^r the Ph'^D i/cfi- the'EckD;. d^i*ee; ^ . 
80% haMaught at the college' (Wei Jp^^t^ee or X^''?/ 6D%%r six'or tno're'Vp^jeprs, (^r>d 
25% for more than-.T6years . * fn odditjdji/ 37^»had kiught far at I'east a )||Oif in e^e^ment^ry 
scfcool and 51^ had a yfear or rhor^Jpf , experience ot the s^cbndary Nevel*.. *Tl^s, mony .Qf 
the faculty wer^'a'cquainted at first hGrxJ wij^i th^-educatippal s^ttif)gs from witich -the . * 
particFpants come and for which' they were being ' tr$ i ned # ^ .i 

Twa sets' of quest/ions bear oV^^the^olity af:l/\stqucti.oh af rhe cliffe^rent ins'titutibrts^ 
' one set from* the evaluatipn teajps^/the.pjm ffOm theTelfows. V^-heo asked ^to comment on 
Ihe qualifioation* of the teachiT)g.'stoff / 28 'ofSf>e.3l^eyaljalJpn'teoc»^ rgfed therrt on the 
"qualified" side of , the continuum, two pla'c^d*|Heif rdtlng'^ at the^ nydpoint, and only one , 
team rated the staff as slightly unqpalified * ^ " 

The* fellows* ratii;^$ of the/rdcaJty,are simMar to j^ose that' would be. given by 
college students in b course that was somewhqt'beHer tHaO average.. Thus,- 6y>^r two- 
.thirds of the ffellows rated" the~quarrty , of Jectuces as'Qobd or excel lent; 56^ gave the sam# \ 
evaluatibfi to seminars.and stru9tured*discbssions. 'Th^ greo't ma^rity of fellpws -said that 
the*, lectures'' v^re seldom or "never over their hedds' (a response^ which might, in fact, be | 
either positive ofNiegatiye), that the. instr^jctbrs did'not talk doV/n to them, and thaf the 
lectures deoTt with >cario^s dpprdaches'to the subject. However,' 41% belreyed thQ|t' lectures 
/ .were sometimes or usually dominated he/ (Retail or u nre la ted* fa qts, and q sUght rrjajorlty 



(51%) reportecj \ho\'jl^e faculty had little Or no knowledge.of the practical probfems of 
school teachers It should be notted that this last opinion does not wholly square with th'^. , , 
facult)f^niembersj reports of their own bacl<grour\d in elemehkary Oftd secbndof'y school , 
teachhig. - \ * ' ' ^ ' . 

• ^ • . ^ ■ \ . ' ^ * , • ^ " * 

^ ln.sum//<+^e instructors at the various institutions seem to have been quite well 
qualified. Reactions of fetiows^tp the teaching were mrxed,, aJthough on the pcJsiHve ^ 
'side. The response to lectores and 'seminars was quite positive; on the otheir hand/ there 
was some fe^Mng — based, ,perjiaps, on.thfe'acaderfiic Qature of most programs'and an the/^ 
fellows 'xoncern about the material's applicability irTt^e classroom i— that not enough * 
jittention was given to the "practical " problems of tea'chers. ' . , 

C. Characteristics of the^Progroms , • , ' * , , ^ 

Th^ guidelines for proposals for the ExTFP emphosiz^d^ en bloc programming, 
coopyration between educotion and subj^ct-mqtter depaTtfnents, and attention tp the special 
character and exper'ienc^ of the participants; the^e requirements, demqnded of progrcun planners 
a typ^t of co-ordin6tiofrthpt may nol" hove been in effect ot many instftutions/ Although 

* the guidelines^ specifically stated, that educationol/nnovdtion was not a reqm'fement 

• for proposals, in fact the proposals whidh were funded ^ere novel and extre^nel-y diverse. 

^ It is this diversity which is most dharocteristic of *the gfbup of programs as a wtjole . Of the ' . 
50 programs, the lorgest number whose titles were approximately similar fs four, and tKere 
seem to be 24 different kfnds of programs indicated by the 'titles alone. Actually,' except 
.fpr the structurol uniformities called for by the guidelines, there was little similarity between \ 
any two programs on more than a few dimensions-. The dimensions along wHich programs ^ 
<oried may conveniently be divided intQ two classes, organizational ond situotiorxal ^ 
charocteristics and (2) goals ohd rationd-le , \ . ^ / 

\ >^ ' \ ' "\ ^ ^ - • . ^ ' 

Organizational and , situational characteristics / The vast majority (84%) of the 
programs were sifucrted., geographically, in one of three areas, tJhe Midwest (38%), * ^ 
West (25°^^)or Northeast (21%)/ The remaining 16% of th^ programs wSere scatfered across 
the Southeast, South CentfoJ, and Southwest regions. . ' ^ 

The progroms ranged in size from 5 to 25. participants, with 2^ the most cpmmon 
number^ (34%), followed by 20 (28%)'and 15 (20%). Only tWq proems had a participant 
group of,5 or fewer. The guidelines specified there should be coopeTqtion between education 
and>fiub|ect matter departments, but in every case o>je departmepit bore primary responsibility 
as "home" for the program. The programs were approximately. eqv^lly divided in their» 
locations, vyith 29^programs based in education departmejits and 21 Ixased in other departments. 



^One reoB^on for the novelt) and cfiversity omong the 50 programs that wer^ctually 
funded may be that the advisory panels used innovation and divjersfty os criteria in deciding ^ 
.which proposals to recommend fqr approvgl or disapproval^ even though theis^ criteria 
weqe not specifically set forth in the guidelines . • . 
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. (j^pof lafifrude wos'permitted to institutions in the actual functionijig of the programs. 
'For instance, mapfy awarded of^jMA degi^,e at fhe^ completioa of fh^ program, others 
provided tbe'^possifai lit/ of dn MA upbn^cbrfipletion of some further work) and some qnpdd ^ 
« no provision for an ad^onced degree at all . Among tKose aWavJing tan* MA degree^ some 

reqpjred a thesis, most-did not. <^1[^ choices of freaching techniques -were related to the goals- 
of tfte programs,, but agafn ihere were great variations among programs with apporently 
•similar goals. Sbme programs, pbrticijlarly those whdse-purppse seemed to be to upgrade . 
^nd .educate tochers in existing areas such as History or Mathematics, provided a menu of 
convfentjonol courses frorn which participants sefected, much as in-a conventionql MA prograni 
Others, particularly those programs which reportedly they were training for a "new" kind ^ 
, of function, such as rft^ia consultant or teacher of the disadvantaged, provided a real 
. "]J^loc" of courses ident^col^F eat:h participant, and often very differefit from any courses 
• Z t^hrelsewl^il^ithe institution.. Seminars, workshops artd practica were in general more 
common in the latr^progrqms than in conventional graduote sequences. ' ^ 

^ks to*the goals' whic^progrdms -pursued, their diversity^has already Been mentioned. 

^Programs ranged from fairly conventfonal, though certafnly important, attempts To upgrade ' 

the cftntent, knowledge, and techniques of teachers of English, History,, or- Geography 

to the creation of a "unique person in the educoUonal setting" such as 6 ceritralized media 

specialist or an educational systems janaiys't . -There were five programs to train teacher? of 

^ different disadvaol^ged groups, •including rural Alaskan Indians, Texas Mexican-Americans, 

and Harlem NegroeL * • ' . . 

^ • */ • , • 

• In 18 p'rograms the principal emphasis was on secondary sihool teacher, in 17 the 
emphasis was^t the primary level, and at Jeast four covered both levels. In addition,^^'ere 
* wereJO pr^rams for the* trainijig of coordinating or advisory perspnnel, suchl3s guidance 
counselors, mecjja specialist^and school librarians: . ' ' . 
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IV. A Profile of Reactions to the Program 



This, section of the report will consider reactions to the Program by fellows, 
faculty members, (;)irectors, and evaluators. It begins with ^he degree of sptisfaction that, 
was expressed, then proceeds to various judgments of the Programjs effectiveness. A 

.discussion then. follows of how and whefher the various programs Implemented the three 
requirements regarding program* structure that were spelled out in th# guidelines: the ^ 

'importance of an en bloc approach^ the necessity for sub|ect-nDatter and teacher education 
departments to work closely with each other, and the requirement that relationships be 
established with the local school systems. Finally, we will discuss what might be called , 
the "strategy of operation" adcJpted by the differen^rograms, including the extent to 
which the programs rnode use of' the extensive experience of the fellows, the atnount of 
competitiveness that was fostered among fellows, their work toad, the extent of faculty 
involvement in the progrg/n, and the amount of innovation in the curriculum. 

A. Satisfaction and Effectiveness ^ ^ * 

• J 

It is not eosy to differentiate between a persoil^s satisfoction with an educatidrjpl 
program and his judgments of its effectiveness. Presumably, the two kinds of responses 
should vary with one another — indeed we shall see in the r>^xt section that they co- 
vary too remarkable extent; nevertheless, the distincj^'on seems'worthwhile, for satisfaction 
witK a program refers to one's overall emotionaT response; a program is judged to be effective 
or ineffective according to whether it achieves the go€illt^t|ie respopbent expects of it. 
Thus; a program might conceival^ly be effective without necessarily producing* high levels of 
satisfaction pmong the participants, and vice versa.- We cCnsidH^ first, the 6xterifoF 
satisfaction with the ExTFP, then judgments of its effectiveness-. ' ^ 'J^. 

• • * . 

\ , Satisfacti o n With the Progt^a'm • >• 

•Two questions which appear to reflect satisfaction with the ExTFP were included 
in substantially the some fo-m in the q\jestionnaires given to the fellows, to the faculty, 
and to the directors. One of these deals directly with reactions to the Program: * 

' ^ * , ' * ' . 

Which of the following alternatives best describes your reaction to the 

Experienced Teacher Fel lowship .Program? 

It was a stimulating and interesting experience throughout. 
A It was usually stimulating and interestir>g . 

It was only occasipnally stimulating and interestir^g. 
It was seldom or never stimulating and i^p|psting. 




A comparison of the responses of the three groups is jgiven in^able 2 . Clearly, 
'the reaction in every group was overwhelmingly favo/able, with faculty members somewhat 
more favorable than fellows, and directors the most^favo/able of alL It should be uncjer- ^ 



rr 



llne^ tfiat among even the least enthusiastic group, the fellows in the program, B2% reported 
fho\ the ExTFP wds either usually stimulating and interesting stimulating and 
interesting throughout. • , , ( « . 



Tabfe 2. Ratings, by f^aculty, directofs, and Fellows of How Sfrri^ulating and* Interesting 
Wey FourxJ the Program 



Source 



Percent Saying Ex^FP Was Stimulating and fnteresting 



Throughout 



Us.uaHy OccbsiofKilJy; 



Seldom or Never 



Fapulty 



42 



46 



T 



Directors 



60 



38 



Fellows 



32 



50 



16 



The second item that may be considered a measure of satisfaction asked faculty 
a<^ fellows for a coryiparison of the Progrpm with a typical eight-week summer institute: 
Faculty members ond dfrectors who hiad taught in such programs, but oot those who hod not, 
were asked to compare the ExTFP' with NDtA or-t^lSF academic year or summer institutes. 
Sirice fellows had earlier reported whether they had ever attended such institutes, it was 
possible to divide them \<\\o two groupsr those with and those without prior instituTfe* 
experience. The four sets of' comporlsons of Thq ExTFP with o^er institutes are presented 
in Table 3. ^ 

* ■ y . - ' 

Again,' it is evident thpt jhe response \o the ExTFP was overwhelmingly favorable. ' 
On this item, the fellovys were mO>e likely^xm the other two groups to view the ExTFP 
as superior toother institutes, and those who had not token part in such institutes were 
somewhat more fovorable than those who had . Again, the directors showed somewhat 
greater approval of the ExTFP than did the faculty. Of greater importonce than these 
inter-group (Comparisons, however) is the fact* that only a tiny fraction of |he respondents 
In any group felt thot other kinds of ^institutes were' superior to the E^TFP. 



It rtiust be emphasized that these judgments are almost certainly expressions of 
satisfaction wjth the ExTFP instead of a true reflection of the relative effectiveness of that. 
Program and other institutes. Many foctors were involved in these^responses: fellows In th^ 
ExTf P had committed o full year orjonger to that program; their stipends were forger than 
those pqio^n the summer institutes and they extended over a full year Instead of eightweeks; 
many of the cVltP fallows were receTvir»g advanced degrees or credit tovyard such degrees. 
In sRort, ExTFP fellows and staff were comparing a pres€;^t valued experience to a distant one/ 
their comparisons can haro|ly be considered unbiased judgments of effectiveness; as expressions 
of satisfaction, however, the results are impressive. 
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Table 3; Comparisons^. by Fa^Py, Directors, !an^ Fellows of ExIFP ahd Other Institutes 



Source • 


Percei# of Group in Eacf) Response Category 


ExXFP 
Superior 


ExTFP , 
Somewhat 
Better , 


About the- • 
Some 


Institute . 
S9mewhat 
Better ^ 


Institute " 
Su perior 


0- 

Faculty with -Instrtute 
ExpedeRce (N=57) * 


28% 


21% > 


1 37% ^ 


9% • 


5%' - 


Directors with. Institute^. \ 
Experience (N-21) 


52% 

J 


19% 


•Wo 


c 

5%^ . 


.5% 


Allows without tnstifute 
experience (N^486)' 


80% 


^ 

^3%' 


5% 


' 1%, ■. 


1% 

• 


FeUowsfvyith Ins'titute . 
Experience XN-pi2) ^ 


i- 

65% 

<> 


. /' 

18% - ^ 




y 


2% 



\ A^number of respohdents omittec^ this item; the totals on whjch the respopses 
are based are those who actually responded. 



2. Judgments of fifpgram^Effectweness' 



Unfort^inatejy, »the questionrKjire method doe^ not<provide a fully satisfat^tory means 
of detentiining the effectiveness of qn educational procfram.-*^SiJbiective ratings can 
provide/ at best, imperfect estimates of whoto studer^ Mas^leorned from a set of material- 
or of how well his new knowledge will be applied when he returps to his earlier role. 
Whether a student has profited a little or a grefbt deal f!*om a program should* be assessed 
by comparing' wh^t he knows at the program's end with what he knew at its beginning; 
sirqjlarly, whether he will apply whdV he has learned can^Be determined adequately 
only by observing his jberfdrmonce on his hbme grounds. Nevertheless, in the absence of 
more reliable measures of progrcwn effectiveness, the subjective ratings that are . 
obtained in questionnaires ore considerably tetter than no esfimates of effectiveness at aU . 
Especially when the respondents are experienced judges of,the effectiveness of educational 
programs — and suchTs cerjainTy the case in the present study" — one can expect their 
rpplies to the questionnoire to relate positively^ if imperfectly, to n^ore, objective meajgres 
£f program effectiveness. With the mpterial at hand, we have no choice but to use 
questionnaire ratings of effectiveness. We cannot estimate the degree to which these 
ratings correspond to the "true" effectiveness of the different programs; nevertheless, 
we can reasorxibly qssume that there is considerable validity in these judgments. 



-17* 



24 



There was only Jh^ question relating to effectiveness^ which osked for'^approximplely 
the same kind of judgment from 'faculty, directors, fellptws, aod eyqiuators. The form the 
question varied considerably from one questionnaire to artol4fer. In the faculty ond 
<Hfectpr questionnaires, 3 respondents were asked: ^ "In your opinion, were the educatiorial 
peeds of the participants met by fhe program?!' Fellows we*re first asked tb' check, 
from among seven different teaching problems, the' one that had concerned them most^ 
before thfey enrolled in the ExtPf . In the next item fhey were osked: "To what 

X extent did the program this year meet thqt problem?" FinaHy/ evaluation teams were ' ; 
^ asked to rdf^^ on a seven-poii;it scale, whether the program seemed to meet the rieeds 

• of the participants. ^Responses of the four groups to these items ore presented in Table 4. 



> Tabje 4, Estimates by Faculty, Directors, FeUows;.and Evalugtors of Whether the 
ExTFP Met Fellows' bJeeds / r .- - 





Were the educdtionaT needs of participants nhet? 




Definitely 


Probably 


1 doubt it 


Not at all 


Faculty 


42%^ • 

: 


49% 


4% • 


0,% 


Directors 


67% 


31% •"' 


1% 

« 


* 0% / 




■ To what extent did the^rogram m6et (your major teaching) 
» problem? ' • 


• To a great , 
degree 


To 0 moderat* 
, degree 


5 To a slight 
degree 


Not at all 

F* 


^— 

Fellows 


\ 
35%- 


' 39% 




6% 




Did the program seem to, meet the .needs of-^rt.iclpants? 


. " We 


NeutrdI 


" PcArly 


Eva lua tors 


i.^ 51% 
J ^ , ^ 


32% ' ' 

• < 


16% 



- It is evident thot the directors and foculty were both confld^t thot the fellows' i 
educationa[ needs 'had been^met, with the directors, once more, somewhot more positive^ 
than the faculty. The confidence of these two groups in the effectiveness of the Program 
'is further revealed by their j-esponses to two other questions. When asked whether _ 
the ExTFP resulted^in th* participants becoming better teochers^ 72% of the faculty replied 
"yes" and 24% were uncertain (presumobly for bck of direct observotion of the fellows' 
teaching); the corresponding proportions for directors were 80%^and 13%. Similarly, 
92% of the'faculty and 100% of the directors reported, in another item, that the overall 
program was either valuable or very valuable for the patticipants . 
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Jt may be seen in Tab he 4 t^KiN74% of the fellows i:!eported that the progrorn 
had' met their, mo jor problem to at least a moderate degree* While ^^is is a substanfial * .) 
•majority, It also leaves one fellow in four feellr^g that*.Ms major teaching problem was * y . 
met to only a slight degree*(21%) or not at all (6%). This outcome shouldt be taken • ' 
as evidence thtit one fourth of ^the fellows thought the program jwas Ineffective. . ^ 
It seems /hore likely that the ExTFP was not specifltdlly directed at the major teaching 
problem of many of the fellows. For example, the major pl^Jem of 28%" of the fellows 
^wos either motivating studentS or handling students of low abilky; it is do^jbtful that mqst * ' 
programs focused their instruction on'those topi<^5ts^ A related, item on the fellpws* questionnaire 
asked them how'useful the program had been in pjiparlng them to jiandle/tJieir^wo 
teaching situations. Seventy-nine percent reported tha't it had bee4i eifher moderately J 
or ejiremely us'eTjul; 18% said it hacl been somewhat useful, and only 3% colled l^he prdgram^ 
not useful at all. ^„ ^* ^ . ; 

I From Table' 4. it appears that evaluators were somewhat less impressed with the 

effectiveness of the programs than were the oth^r three groups. Just over half of the 
e;/aluation teams said the. institutions they visited hocj met particip^ints' needs well^^ while- 
five teams said these need^were less than 'adequately met. These Id^^ive teams of 
evaluators remarkeltl on the similarity of the programs they visited to;^gular undergraduate 
and graduate training, and also on the lack of adequate practicum^experience , A much 
more favorabrle view of the programs was expressed in evaluators' judgments of whefner 
the fellows would be able to apply what they had learned when they returned te their 
schools. Twenty-three of the 31 teams reported in the affirmatfve, four placed their ' v . 
ratings at the midpoint, and only four teams said that the students they observed were 
somewhat unlikely to be able to apply what they had learned. Several tearu§ remarked 
that their judgments were less favorable than they might have been because they feared 
that traditionalist or mon^-consciouls school systems might resist the introduction of 
some of the materiel the fallows would brinaback wTTR them from their year of training.- 
A final indication of evajuators' views of"the effectiveness of the Program comes from an 
analysts of th^ general comments they wrote at the end of the evaludtor^ form . These^ 
comments revealed a clear acceptance, by all evaluators, of the general value of the ^ — 
ExTFP. Even those few evaluation teams which,expressed rather extreme criflckm of 
the institutions they visited felt that the fellows had'profltecpin some cf^gree from their 
year of study; their criticisms frequen/ly stemmed from the convK:ti9n .that substantially 
more could have been accomplished had, the program been conducted. differently . ^ 

In Summary, it is clear thdt there was general satisfaction with the Program, and 
widespc^d agreement that it was an effective educational venture. It should riot be 
surprising that the directors,^ faculty, and fejiows expressed approval of the Prlferam. ^ 
When one devotes a full year to o project, there develops considerable internal pressure^ 
to view that project favorably. ^ Despite this built-in bias, the overwhelming favorableness 
of the opinions given by these flyee sources strongly supports the conclusion that the 
ExTF(^*Was a satisfying experiepce for fellows and staff alike. The evaluatibn teams 
had no personal involvement in the outcome of their evaluation. ' They* were specifically 
assigned a critical role, and they measured the programs against high standards of success. y 
Their generally positive evaluations provide further evidence of the program's effectiveness. 
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B*. Struafural Aspects of the Programs • , ' 

Dfcubtless because of the lack of specificity in the guidelines, there' was considerable 
variation in the pcj^ttern of organization adopted by different institution? in implementing . 
^the ExTFP/ Sinte only^o limited number of items dealt with such questions/ many of- " 
these differences in program structure went unrecorded. Only l^ee structural aspects ^ 
bf-the pgograms will be dealt with here, all of them specifically^ discussed in the guidelines 
for proposals* the utilization of the; en bloc approach, the relationship among teacher- 
ifducation and content deportrnents, .an3 the ppttern of relationship with ^ocal school systems.. 

1. The en bloc Approo^, Group Solidor;ity, and Morale 

Only one question deal t^ specifically with whether the en bloc approach was 
adopted by the various, institutions; this was an item in the Vrsitors Evaluatioa Form which 
iaid "Unlike conventional graduate* progrqms, th^|(bcperienced Teacher Fellov/ship Progroiri . . 
Is based upon a block'Or group program^appfoach . The intent is to, use the group to 
er^I1ance learni^ by building morale and esprit de corp3% Has this been successful?" ^ ^ 

^valuators' responses make it clear^Vliat the en b^oc approach was, indeed, 
successful: 12 evaluation teams reported thatthe approoch' was- extremely successful an^ 
13 others rdt^d^he approach as successful, but not e5c+remely so. Only three teams said 
tlxit the approach was in some degree unsuccessful. 

Although the en bloc approach was not mer\tionefl in the other three questionnaires, 
all three groups were asked whether there was a feeling of group solidari^ among parficipants. ' 
' in^theJpVocjrqm; in addition, respondents w.eVe asked, to,rate the overall morale of the 
participants. ^The responses of tb^ •three groups.ore presented in Table 5. 



Table 5, Ratings by Faculty^ Directors, and Fellows of Gfoup Solidarity and Moralge . 
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a. Was there'a feeling of group solidarity? 
Percent Answering ' f • 


Sour'ce 




Strong 


• Considerable 


' Some 


None 


"acuity-- 
• 




' / 53 • ' 


38. 


.5 




pir%ctSrs v 


67 • 


27 


■ 7' 


-0, 

»• 


Fellows 






14 


^1 • 






b. Howwould you rate felloyvs' morale? 




Source 








Average , 


Low arid 
Very. Low 


— 1^ 

Facufty 




-23 


•51 ' 






i< - ■ 

Dire^ors' 




38 




• 1? ' 


H ^ 

- 2 .- 


pel tews 


"' 11 


38 ■ • 


27 


IP--. ... 



^ 27 : 



'•^^ , '♦B/ every measure, solidarity and morale were high. Again, the directors' 
estimates of both variables were^somewhat higher than those by the faculty, and the 

^ -facullfls. estimates wer^ higher than those by the fellows; but even among theiellowi, only, 1% 

said there wfts-no feeling of solidarity cfhd only 10% reported that the group's morale' 
' ' was below average. It should be mentione*d that there was Considerable homogeneity 
withlfjiCgroups of fellows in theifr judgments of solidarity. That is, reports of relatively 
low solidarity were concentrated in particular. institutions; they were no.t made by 
social Isolates scattered among a number of programs^ut probably reflected, instead, 
a real lack of solj^arity in a few of the programs; ^ * ■ = 

2 . C ooperation between D epartme nts 

^ It will be recalled* that the. guidelines for the ExTFP specifjcal ly called*for cooperation 

between subject- mofter and' teacher-education departments in co'nducting the program. 
Three questions, one each from the fellows, thedirectors, andjjje evaluatOFS questionnaires^, 
askefl whether such cooperation was achieved. ~ ^ 

1. • 

In sope institutions,: such cooperation apparently tUd not extend much beyond 
con$ultation>on( fhe initial application. When asked whether the director of teacher- t 
- -^iriucatian Avas involved in fie operation of the ExTFP, only 26% of the program directors 1 
replied that he was either quite involved or very much mvolved;; just. ovpr half said he ^ ^ 
was not very involved, and another 20% reported that he was not involved at all. 
This question, of/:ourse, aslced only about th^ director of teacher education, not whether 
there was cooperation with dthers in^esslr positions in the teacher-education hierarch)?. 
In fac*, when they were asf:ed to describe the cooperation they received frorrCoither 
* academic departments,^ 91% of 'thS directors said it was eifher quite good or jjnusually ^ 
good. This suggests that some dol laboration must have taken place between teacher- 
edudotion ahd subj^t-matter departments, or at least that the directors thoucjht so. 
Evaluators.' reportJ indicate that interdepar^ental cpoperation varied widely fr^om one 
institution to .another. While only one evaluation team reported very close cooperation 
between the two. departments, ahother 12 pkaced their ratings at the cooperative side of 
tKe'continuuym. On the*other hartd, 15 of the evaiuatoi's' ^dgments were on the uncooperative 
side; in five of these institutions evaluatofs said fhere Was no cbopeTation at all between 
teacher-education and subject-matter depa^ftients. ; 

Responses. by fellpws show much the same picture as those by evaluators. Eighty-four 
percent-of the fellows* reported that their instruction involved more than one academic 
department. Exactly half of these, 42% of the total group, said the material was coordinated 
either quite well or extrem^ly-well; the^ other half, again 42% of the total grbup, ,5aid the 
. material was either not coordinated too well or was not coordinated at all . We should 
recogfiJze, however, that tfii^ question does not^bear directly ^n*the point at issue, for the 
second depprtmept vvhL^h the fellows had in mind need not have been the department of 
teacher education. .One qtj^er item on the participants* questionnaire had at least a 
tangential bearing on this question. In response to a question about the relative e'mphasis ^ 
on subject matter'and teaching Miethoc(s, the majo/ity of fellows, 64% said that the bplance ^ 
was about right, 28% reported that tfpere was too much emphasis on subje^ct matter, and 
only 3% reported too "much emphasis on teaching methods. 



3. Cooperation with Local School Syste;^ 

' Only one question deoft with this topic: evaluators, were asked to report how 
extensive the relatiorjships were between colleges and universities and cooperating 'school 
districts. Again, there were great differences from one institution^o another: fourteen 
Institutions were rated on theMow end of this continuum, 1/2 at the high end, and five at 
the midpoint. In the'^aluotors* written cornments about th^-^ograms, some of the most 
causti'C had to do with the^lack of practicum experieno^^^Jvai bble to fellows in those 
institutions without relationships to cooperajive schyf6Tsystems . 

Tp summarize, the en bloc approach seems to hove been effective in most of the ^ 
instih^ions that porticipoted in the ExTFP;'it was accompanied by a high degree of 
solidarity in most of the groups, tind by reports of high morale among the fellows. In the 
motter of cooperation between teacher-education and subject-matter deporfments, the 
program does not come off so well: in some institutions there was, substantial pooperation 
of this s^ort, in others there was little or none. The same wide range held true for the- 
extent of cooperation between the participating Institutions and local school systems. 

C. Operating Strategies % • ' 

Even when programs have the same formal structure, there moy be extensive differences 
in their mode of operatidn along a variety of dimensions. For example, graduate programs 
in American universities are known to differ rn the degree of competitiveness thai they 
foster among graduate students; it would be expected that»ExTFP programs would also vary 
in this regord. Similarly, frj»fversittes, and by extens/tn the ExTFP programs, differ widely 
in the extent of faculty involvement with.sbjdents and in faculty commitment to instruction. 

Bevond this, the nature of the ExTFP suggests that there are other^'merjsiOTi/* 
along which ^cirJation may be expected. The experiience, of teoching for g Ttumber of 
years has provided eacl> fellow in the P^ram with o degree of specialization in hjs field, 
an awareness of the problems thot are irr^olved in teochtng his subject, and q,n intellectual 
and emotional maturity that set him apart from the typical graduate and undergraduate 
student: His response to the program of courses that is offered should depend, in considerable 
pdrt, on wl^ether and how* that program builds upon and utilises his extensive experience. 

We turn now to a discussion of differences among institutions in these aspects. ^ , 

1 , - Utilization of Fellows' Background 

y ' - 

All four sources — feljows, fqculty, directors, and evaJuators — were asked, in 
• one woy or another, whether they'felt the program had taken .advantage of the rich experience 

njj^jrTnr preporotion'of the fellows. Although these questions wfere phrased differently ^ 
, iCr^fferent populotions, the four sets of responses hove been grouped -so that they are • 

roughly comporable; they^ore presented in Toble6^ ^ 



Table 6. Judgments of Whether Programs Built Upon Fellows' Experience and Pfepardtion* 




— ^ — * 

Source ~ y 


- Percent, of Each Gr^up Respondfiftg 


■^Ye» ■ ^ 


Unceriain 


No 


FocultyjBg^ . ' / 




' r 

23 


. 6 • 


Directors 


89 


\ 

7 . 


- 4 _ 


Fellows 


63 ' ' 


/ 

24 


12 


— . 0 *— 

Eyoliiators* 


'32 


•16 





In the faculty and directors questionnaires, the alternatives fof- respondents 
to check were 'Yes/' "Uncertain, " and "No, " Fellows who said that frhe progrqm 
usually. or consistently built on their backgrounds have been stored as replying "Yes, 
those who said it rarely^id so are scored as'"Uncertain, " and those who said it was 
unconcerned for their background are scored as saying "No. " Evaluators' responses 
are recorded as "Yes, " "Uncertain, " or "No" acc^fding to whether their judgments 
were on the positive side of the ^midpoint, at the midpoint, or on the negative side. 



Jl Xlearly thtere was a considerable difference among sources in their judgments ^ 
of whether the programs^ took a^ccount of fellows' backgroupcU . Program directors were 
most likely to,say that the programs had built on fellows' backgrounds; faculty members 
and fellows were somewhat less certain, but the clear majority of these two grojps agreed' 
with the directors that the progtam utilized the fellows' prior experience. Real disagreement 
with these judgments was shown by the evaluation teams, over half of whom said that 
the/institutions* they, visit^^ had not designed their programs to take account of fellows'' * 
experience. The comrrfents of those tedms that were Cfitical, on tjjese cyjiunds, of the institutions 
they visited were examined in's6me detail . In an interim report, based'^fc about half of these 
responses, comments by critical evaluation teams were summarized in a manner which 
holds tjue after the remainder of the data have been collected: 



Most coM^nly, [evdiuators wKo were critical] remarked that {lelfows 
were treated likdjPgular graduate students, complete with the Institution 
•of multiple-choice e^^aminations and competition for lerter grades, 
with the prescription of a fixed schedule of courses, with little tailoring of 
individual programs to the rieeds of irKlividual felloiys, and with . little 
'or no opportunity for, fellows to exchange ideas with one gnother about 
„rtielr own experiences. In short, . , .these institutions offered Substantially 
the same kinds of programs they had always offered. 
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Why the other three "sdurces at those same institutions should so strongly dTsagree 
witF|||fhe^e valuators' judgments remains unclear. Perhaps in their involvement with one / 
iristitutic^n, they were unable to conceive of thctrange of alte/notive policies thof the 
evdiuators envisioned; -d!^ consequence, policies that these sources thought were major 
concessjbns to the fellows* experience may have been viewed by evaluators as modest 
efforts, <jtl>e5t. That faculty members and program. directors actually did believe they 
had utilized the fellows' experience is made evident by their replitt to a question asking 
whether their program was modified to take advantage of the experience an?) bockground 
.of participants. 5ixty-two percent of the directors and 36% of the faculty answered 
this question affirmatively; 9% and 40% were uncertain, and only 27% and^ ]6% answjered 



•no.' 



Competitiveness and ^ork Load 



w Our interest in discussing the 'extent of coijipetitivene^ and the size of fellows' 

work load is not to determine whether^participants in the ExTFP worked or loafed, but 
to see whether they thought they were overworked and how- intense was the competition among 
fellows. Unfortunafely, the question that waxgsked of faculty members and directors 
qppears to have Been relevant to the first question, not to the second. It asked whether 
the students worked hard during the year. Eighty-four percent ^jl>e program directors 
and 79% of the faculty replied "yes, "jTBare 13% and 15% respectively of the two groups 
said "yes, too hard, " and the^remaining few respondents-were uncertain or reported that 
fellows had not work^sd hard. In retrospect, these replies seem to Veffect the respondents* 
approval of f^llows In their programs more than'their evali^tion of the fellows* work load. 

By contrast, fellows were about evenly divfded between the opinion th&t'tReTF 
work load was about right (52% of the r^r^pondents) and the view^hat it was too heavy to . 
allow completion of assignments and independent work (47^o), Rotingsby the evaluator ' 
« teams also indicated that the work load in some schools was heavy: ^0 of the 31 |eams 
reported that the wferk load at the institutron/tliey visited was about right, the rerpaining 21 ' 
said that the work load was in some degree too heavy.' As to competitiveness, 6Q% of 
the fellows said that the level of^competition in their program was either quite^high or 
extremely high, 2\% soid it was about right, and only 8%, said it was either low or very low. 
It should be mentioned that there was considerable hornogeneity of judgments on these 
items among fellows in the same programs; -that is, in certain, programs almost all of the 
fellows said the work load was too heavy, in others, almost ail sakijf^^s obout right: 

* H A cogent comment on these judgrtients is the remark that graduate education • 
involves a great deal of work wherever it occurs. Indeed, rnany^-of the evaluation teams 
who rated tfje work locrd Os somewhat too heavy observed that such is the norm in graduate 
school, and that^after the program was ove* fellows might cherish their 'exp^iiw^ce the 
more for the fact that strenuous demands had been mode of them. Neverthelesi, it 
appe^r^ that some institutions did require far more work than their fallows could produce^, 
thereby introducing severe emotional stress Into the academic program. 
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' ifitssi^nment to teach in the ExTFP might have'l^n accepted by g faculty member 
as simply another uni.tin his teaching- load, requiring^© change in the kind of material 
he presenfed, in ttie,way he presented it, or in kis involvement with the students. 
Alternatively;Vlt might have been viev/^d'as a special challenge which callfed for a 
somewhat different orientqtion tov^rd both the subject matter and the students. The ^ 
tone of the guidelines m^es it clear that their intent is for the second attitude to be 
dominorvt ampng the staff of thefxTFF.^ There were no items: in any questionnaire that dealt 
directly with the mode of orientcftion of the faculty, but there were.a number that skirted it; 
we turn. our attenfTon now ta tKose items. ^ . ' ^ , 

>* Facukfy members and program directoi^ were oiked whether jhey fourxJ the ExTFP 

clKillenging ond.satisfyinq experiences. As may be seen ir^ Table 7, 'the majority of both 
grqups replied in tlie affirmativve' to both questions. Prcfigram directors, once again, were 
*somewhat more enthjsiaJtic thon ijjpe faculty . ■ ^ 

Table 7.' Ra tinges 'by. Fdculty and Directors-of Whether^the Program^ Was Challenging 

and Satisfying » - ^ . * 





• 


Q How Challenging was the ExTFP? 


Source 




Extremely 


.Somewhat 


Not' ye 


Not at AM 


"acuity 




47 ' " 


42 




' . *■ . 
0 


Directors- 






) 3l" 




0 






.b. HoV satisfying vAas* th^ f xTFP? ^ , 


Source 




Extremely^' 


Somewhat 


■ Not Very. , 


Not at All 


•acuity 


0 


. 57 


40 : 


3 


\ 

V ' 


directors 


71 - 


24 




0 



The erYfhusiasm of the dfrectors and faculty for teaching in the program was 
cfearly picked up by the evaluation teams, for 22 of the 31 reportedVhpt the directoT 
and staff *were challenged arxl fctrmulated by the program; only 8 ga/e judgments that 
fell toward the opposfte pole of the continuum. L ^ 

. FeNpws were not^ked about whether the sta^ was challenged by the program, but 
whether they were accessible and helpful to- students. Their responses yver^ overwhelmingly 
favorable on botk counts; 94%'i;eported fhat the staff .was either usually or alv\^ys accessible 
95% saidjt was either usually or always helpful . ^ . 



Althbugh, os we hove seen, the foculty ond directors reportfd being challenged ond 
stimuloted by the Proafom; they .were less likely to report that the^^experienpe affected 

^their own pfofessionaf orui iwtfellectual growth, As moy be-seen in Joble 8, only 41% of the . 
faculty said their professional devJe^opmenf was furthered either "greatly*' or "yery greatly" 
by the experience; only'obout 30% soid it odded greotly or ver)^r€^Hy to their intellectuol 
growth oofl >o their ski IJ os teacl-^^^ Agoin, directors were more g^rterous In their 

► esti«t)ates of how much. benefit,they||derived From 'tJie Progrom; 51% sbid it: odded greotly ^ 
or very greatly to their professional gr^i^vth, but only obout o thfrd judged it had o j 
corpparoble effect on their intellectuol growth or their skiHs os teachers. In each cosej 
respondents* judgments of the benefits they derived from the progrom i/ve re less fovoroble 
than their ratings of the challenge ond sotisfoction they feft. ]f is not cleor how much 
weight should be given to these results. Perhaps the. experienced University teocher does not 

, ordinarily, profit in thes^ ways^from his teaching experience. In ony cose, it is clear that 

, tM^explSrTence may hovfe been stimuloting ond challenging but was not viewed os educational, 
for the*majority of the stoff . ' , , 



V 



Toble 8. Judgments by Faculty ond Directprs of the Progrom's Effects on Thejr Own 
Development • ^ 



Item 


Source * 


>^ Percent Rdspo.nding ' 


Very 
Gr^BOtly 


Greotly 


Moder- 
otely 


Little 


Very 
Little 


2&. Add to profession 
growth ond develop- 
ment? 


Foculty » 
Director 


.14 •• 
18 


'27 
36 


35 

42 . 


14 
2 


5 
0 


29. Add to your intellectuo 
growth ? 

w 


Foculty 

Director - 


11 
9 


19 
29 


40 

56 ■ . 


18 

■ 4 


7 

0 ' 


30. Add to your skill OS 
0 teocher? 


Foculty 
Director 


10 

9 ' 


19 
24 ■ 


42" 
53 • 


19 
4 . 


4 

4 - 



^ercentoges in eoch row do rK>t tofol to 100 becouse non-respondents ore nof 
included. & 
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4 , Departmental. Ihnovoti veness • ' ' ^ 

It was not required that instftutions- prepare thorqughiy innovative proposals^ iij • 
order that their programs be funded under the ExTFP., Instead, substantially ^traditional 
proposals were examined ir^ompetition with completely innovative ones; approval or ^ 
disapproval for funding was not determined in terms of the program's novelty, tut in terrfls 
of how effective it seemed likely to be in furthering the education of Experienced teachers. ^ 

In the vieAvof prbgram directors, ihere was innovation in the great majority, of 
^hfe institutions. Of 42 directors who res|3©nded to this item, 32'6aid they hod seen 
imaginative teaching methods and practices in their programs, 6 vy^re unsure, and ortiy 4 * 
reported that they hod not seen such practices. As usual, judgments by faculty merftters 
were less extreme: of(l74 wKo responded to the item, 79 reported innovations, 22 we<lp ' 
unsure, and 73 reported none, * ' ^ ^ 

The e.valuation. teams ogreed more with the fo^Sp )jbon wUh the directors:* 
12 teams rated the, institutions they visited as being on the innovptive side of the midpoint*, . 
'12 ratings were on the noninnova#ive side, and the remaining 7 were exactly at the . 
midpoint. None of the evaluotor ratings fell in the_m'6st extreme categories, those which 
indTOted either a great deal of imagination and iffnovation or none at ol I , 

In summary, there was disagre^mfenTair^ sources ir^the extent to which they ^ 
thought the programs hod utilized the background and experience of theTr fellows; the _ 
least enttiusiqstic source of ratings, the evoluators, 'judged that there were more 
. insftitutions which did not make sufficient use of the fellows' experience than there were 
which did. Apparently, there were systernatic differences between institutions in the 
omourit of -work They assigned their students: *every Jrtstitution required a considerable 
amount of work, but some assigned on enormous amount. By^oll Ofc^ownts, the faculty 
was challenged ond'stimuloted by the program, accessible and helpful to thej fellows. Finally 
it appears that programs were neither thoroughly innovotional nor stodgily traditional . 

5, Effects on Institutional Development ^ 

One of the benefits the Program might have wrought^ irdeed,^ one of the effects 
that was envisioned i^nitiolly, was a strengthening of 'the participoting institutions themselves, 
particularly in thetr on-going teacher education 'programs . .Obviously, changes in the 
pattern of teacher education will hove effects ypon the preparation and later performance of 
those who ore trained; therefore, it is importoriV to determine whether the Program actually 
influenced educational patterns in fhe host institutions, Impcfrtant though this questtor) may 
be^ it is uncertain whether il con bp answered adequately by the present study. These 
dato were collected in the first year of th^ Program's operation, barely eight months 
after it was instituted . * Whatever effects it may ultimately have upon procedures for 
teach^ training, these effects ore not likely \d have taken ploc^ by the time these data 
were collected. Consequently, the Conclusions we may draw about such effects >fnust 
; inevitably be tentative . , • / 
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' Of the five iteite that dealt >^ith this issue, two were global judgments', by. the 
directors prxJ the faculty, of thefrogrom's "value to the institution." One evoluator hi 
itfem qsked about the program's contribution to the plans of the Departmer>t, and two . 
Items, from th^ evaluators and thfe directors, asked specificc^ly about the Program's 
"impact on the on-going tgjfc^herreducatlon pfogrant. " Thus there were'three distinct, if ^ 
related, issues involved in these items: value to the institution, contribution to department 
development, and impact on teacher education^ 

• . • *« 

As was so often th^ case, judgments on these questions varied remarkably from one 
source to another (Table .9)T^ore than. three fourths' of the faculty members and the directors - 
reported tha^ the Program was either 'Valuable" or^ "very valjable" to their institution. 

^ On the other hand, among evaluation teams as many said the program's contribution 
to departmental development was slight or non-existent* as said it Vas'^moderate or great 
(36%, in eacb case). The same divergence of opinion was found in judgments of the Program's 
effect on teacher^jedycation: 62%-6f,^the directors said its impachon teacher education 
^s relatively large^/xl only 23% said it was smdil; tKe corresponding figures fof evaluation 
teams were 16% and 58%. . , " - ^ 



Table 9. Judglfnent^y Faculty, DirecTors^ and ^vdluotors of the Program's Effects Upon 
Ipstifutiona'l Development 



Source 


a. Perc'ent saying Program's value to Institution^,^ 


-Great 


^ Moderate 


Undecided 


Slighf 


None 


— ^ * # 

Drrectof*v^ 
Faculty ' . 


42 . 
•29 


44 
• 47 


9 • 

13 


• * 

2 


0 
1 


Evaliiators 


,b. Conti;H5ution to departmental and institutional development 


• • 

26 


10 


29 " 


26 


10- 


Directors 
EvoliXstors 


c , Impact ( 


Dn teacher educotion 


24 

\ 6 


38., 
10 

p '• — — ' 


13. 
26 


16 
26 


7 
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'To make results from three questionnaTres co/nparpblfe, res»ponses to the Visitors . 
Evgjjkjotion Form have been classified as follows: qjiecks in either of ^the two most, favorable 
categories ore scored as judgments that the program had great effects; those in the third most, 
favorable category are scored as reflecting moderate effects; those in the center category 
are classified as "undecided"; and responses are considered as-imputing slight" effects or 
none according to whether they fell in the third or in the two most unfavorable categories. 
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* These differences in judgments by the different sources probably reflected a number 
of factors. For one thing, the di^ctors and"^e faculty had more at stake in the Program 
than did the evaluatori; no doubt this involvement inffuenced their judgments in a favorable 
direction . It is probably true, as well, that the different sources used different criteria 
to assess the effects of the Program; what looked like a remarkable advance in the context 
.of a particular institution may have seemed trivicJ to an otjiside observer. Paradoxically, 
spme of the disagreement in judgments may have^occurred betause institutional changes 
had been made before the'PxTFP was undertaken: a few' evaluation teams sdid the Program 
had little impact on teacher education because the existing procedures were advanced qfid 
effective; directors at those institutions attributed more influence "on teacher training to the 
Proaram than did the evaliKitors. 

One determinant of whether a change was effected in the host mstitution s t.eocher- 
trainiftg practices was the division of the institution in which the ExTFP was located.^ All# 
five of the institutions that evaluation team} rated above the midpoint on "impact on 
teacher education" wer^ b#sed in education departments. Judgments by evaluation teams, 
of the exte^nt Jo which departmental and institutional development were affeeted shewed the 
same patterns: 7 of the 8 programs in which evaluation ieams sar^ the effects were greatest 
were based in education departments. Not surprisingly,* then, educational chonges were * 
more likely when Deportments offducation were directly»responsible for the Progrpm; 
stated somewhat differently, educational programs that were located ii^ Liberol.Arts 
departments did not have immediate effects on the^policies of educatiorj departments. 

D. General SurVimary of Impressions of the ExTFP * . • ^ 

♦t is clearly evident from the results that have be«n reported in this section 
that the sour^^' evaluations of the ExTFP varied directly with their involvement in the 
program. Program directors, who probably had the most at stake in the enterprise,- 
were thoroughly eiJhusiastic, not to say PollyannaTsh, in their ratings. Regular faculty 
members and fellolWwho were somewhat less personally involved than the directors, made 
judgments that were a little less enthusiastic than, those of the dtrectors. Evaluiitfen teams, 
who spent only two days viewing the programs and who maintained calculated objectivity 
as theiL ideal, were able to temper their enthusiasm with criticism. 

Yet it is the burden of this j-eporHthat all four sources, including the evaluators, 
produced predominantly favorable judgments of the progtom'. Furthermore^ the responses 
of those who were involved in the program are not to be discrei^lited simp|y because of their 
involvement. In all but a very few institution^, it appears, a group of highly qualified _ 
tiKichers were brought together with a group' of intelligent,' hard working, experienced, 
thoroughly committed students* When cirdumstances also promoted the-devefopment of 
strong gVoup solidarity1ar>d hi^ morale among the fellow^, a truly impressive educational 
experience probably occurred. Even when the social context was less than ideal, the 
Juxtaposition of a 'first-rate student body q^nd a better-thap-competent faculty doubtless 
pr6duced educatio/wl effects that were considerably above the average. ^ 
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V. ^Correlates, of Effectiveness and Satisfaction 

have seen thdt satisfaction with the ExTFP and judgments of its effectiveness 
were both very positive. Nevertheless, there were consistent differences from one program 
to qr>other in the extent of satisfaction of the various respondents and in the^ ratings of 
effectiveness ttsL^'hey gave. Our purpose in thisr section 15 to examine the relationship 
of other voriablesjo judgments of effectiveness- and satisfaction. ^ 

Two sources of evidence — ome quaiitative, the other'quantitative — are used 
in this analysis the qualitative material consists of comments by evaluafTon teojns about 
ihe influence that program directors had upon the effectiveness of the ExTFf. This chapfer 
begins with a discussion of these comments. The quantitative material is mode pp of 
correlations among responses to the various questionnaires; the analysis of this materiaL, 
constitutes the bulk of the chapter. A detoiled summary is presented at the end. of the 
chapter; readers'-who are unfdmijihr With correlational materials may find, it helpful to 
read this summory before examining the cgrrekition tables in detail t 

A. The Role of the Director in Program Efte^^eness 

Anoiysis of the role of the prograir's director in the operation of*tbe ExTFP was 
nof systrfhoticallylbuilt into the questionnaires and the evakwtor'i ratings. Nevertheless, 
sports from evalyation teams mode it clear that the actions of the*director were frequently 
crucial to the success or lack of success of indivijlual programs, Onde this became clear, 
the writt^en comments of the evaluation tearps were examined in detail to make whatever 
inferences were possible about'iJiis topic: Analysis of these comments may be summarized 
jds follows: ^ 

' It is apporent'from the reports of evaluation teams that the quality- of directors 
had a major Impact on the conduct of programs. In general, when the evotluators 
commented on*the ability, dedication, enthusiasm, availability, and seriousness of 
directors, they tilsb rated the programs as effective and fy:oductive. When comments * 
were made about the director's lack of status in the institution, when the directt)rship ^ 
changed betwetfrV the time of application pnd thp time the program began, whSn tension 
ruled betweeh th^ director and his staff, the program was characterized as weal^ poorly 
picmned, poorly integrated, and unfSroductive . , 

■ , - ■' \ • I ■ 

Because there was no ,provisi<dn in the guidelines stipuloting that' the director 
be given released time for his administrative duties, many directors lacked time to 
parry out their duties and lacked fundsior necessai^ supporting work. In some cases, 
the director functioned as a coordinator rather than on administrator, with rwither the 



^Professor WilJiam Engbretson carried this analysis and drafted the summdrizfrig 
statement . ' * • ^ 

/ - 
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power nor the funds to cortduct the program as it hod been represented in the proposal . 
lo surpV programs seemedTx) be most'effective when the' director' was deeply inyoTved^ 
in the progrdWi's goofs and was able both tcJ devote sufficient time to administrative 
.duties and to foster cooperation and respect from participants and faculty. 

The frequency and urgency of these comments suggests that special consideration 
,should be given to this key role in.future studies of the ExTFP and in the orgbnization of 
indivrduol progrcfms. ^ * » ' ^ 

B. Some Comments on Correlational 'Methc^ % 4!^ ^ 

1 ^The Noture bf the Data , • , ^ / 

», • 

^To determine the relationships among variables, for every institution the arithmetic 
mean was cpmputedfor the judgments made by each'^ource on the items that were of 
interest. This permitted institutions to be arrayed, for example, according to the average 
degree of sotisfaction.that the fellows expressed, according to the average faculty rating ^ 
of effectiveness, and so an for a substantial number of variables. Product-moment' 
correlation coefficients were then Qomputed among these variables. 

It should be obvious that two variables cannot be correlated with one another 
unless there is at least some Variation.irr the scores on each from one observation .to another . 
^If aM of the scares .on one item fall atHhe identical point^ then responses to that, item 
cannot possibly co-vary with responses to some other item. On mariy of the items that 
dealt with satisfaction and effectiveness the responses of directors showed next to no variation 
being largely concentrated at the'most favorable alterna'tives. For this reason, directors' 
responses will not be included in'the correlation matrices that are presentedTThihis section. , 

There remained responses by facuJty and fellows at 47 iffstitutions, and responses 
by faculty, fejiows, nnd evaluators at^31 institutions which wer6 visited. .It seemed 
clear tFKit our interpretation of tJje results would be substantially strer>gthened by 
including a discussiorv of the correlations of evaluators' judgments with those made by 
fellows and faculty members. Howeyer, correlations based only on the 31. programs 
that were visited might, because they ignored 16 other institutions, ^ive a distorted picture 
of the true pattern of relationships among variables . To make sure tha/ this was not the 
cose,^ two correlation matrices were computed, one based on responses by f^j^culty and 
fellows in all 47 institutions, the oth^based on responses b>r faculty, fellows, and . ^ 
evaluators in th^ 31 schools that were vi&ited. A comparison of the correlations between 
identical pairs of variabiles in the two matrices showed that very siirwbr resultsj|ere obtained. 
Therefore, only correlation coefficients based on the 31 progrorps that were vislfed will be 
ysed in the results that are reported below. With a set of 31 observations, a correlation 
of about .35 is required for the inference that it differs from zero by an amount greater 
than would be expected by chance. ^ ^ 

r 
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2. Interpreting the Correlation Coefficient | * , ' ' 

When two variables show a sizeable' correlotfon, it is often tempting and sometimes 
reasonable to conclude thaj* they are, somehow causally related.. This temptation should 
be Indulged with caution, if at all; causal relationships cahnot be established by corfefotional 
techniques. For example, we shall see that there^was a high positive correlation between 
fellows' morale and their judgments of pfbgrofinref^ctiveness; in programs wHere morale was 
high,, fellows' judgments of effectiveness was high, when morale was low so were feUows"*, 
judgments of effec>fveness . 'Clearly, however,, this does not mean, that high morale 
produces an effective program. It is equally -likely that the causal chain goes the other way, 
that morale g^s down when a program becomes i^neffective or up as effectiveness improves. 
It IS also plausible that the two variables interact, so ftiat some degr^ of iheffectiveness { 
depresses morale, which makes for even less effectiveness, decreasing morale still further. ^ 
The point is that one should be cautious in interpreting correlations. The results that will 
be reported below often seem to point toward ways by which progrdms can be ii^proved; we 
believe, in fact, that they offer^suggestions for iiTiprQvemen^ . But these suggestions, must 
be examined intelligently, not accepted uncritically as-a consequence of an imprtSssiyely large 
correlation coefficient . • . ' . 
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A final point must be made. It has long been known that when judgments are 
mode on several variables, all of which have a desirable and an undesirable pole, a 
built-in correlation is introduced. A respord^nt-who tajces a favoiuble or unfavorable 
stance with^espect to some issue is likely to rate all of the subsidiary aspects of that 
Issue in a manner consistent with his ovef^-all position. In particular, people who are - 
favorable to the ExTFPos a whole would probably be partial to all its' parts. We have - 
already seen evidence of such a tendency in the responses of the program directors. So 
a certain degree of correlation must be expected between any pdir of items from the same 
questionnaire as a simple function of -this bias. However, sgch a bias carmot be invoked 
as an explarKition when items from different questionnaires correlate >o/ith one dnother; when 
two different sources agree in their ratings of«an institufion on some dimensions, the 
' bia^i^f either source alone caftnot be invcrked as an explanation. * for this reoson, special 
attention must be given to the correlations between judgments tha* were made by different 
sources : ^ 

C^ Correlations Among Measures ofrEffec/fveness and Satisfaction 

*' * ' » , 

It has already become evident that faculty members, fellows, and evaluators 
ail showecl'^avorable opinions of the effectiveness of the ExTFP. The question at hand 
is whether an institotion that was rankeJ high on one measure also received a high ranking 
on another. For purposes of this presentation, measures of satisfaction and effectiveness ^ 
will be combined in one correlation matrix owing to the fact, as we shall see, that the 
'two kinds of measures had very high correlations with*one another. 
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Table 10 presents the f>bttern of correlations ameng eight measures of effectiveness 
and.satisfhction. It is apparent that there were consistently high positi'ye correlations 
among such ratings when they wef>e given by the same source. Thus, the average correlation 
among the fqculty items was 4. 55 qnd the average correlation among the three fellow 
Ittms was 4. 72; High correlations were al^o^btained between judgments by the fellows 
and those by evatua tors/ the average correlation being 4. 55. As to correlations between 
faculty judgments and those by the other two sources, however, only focuky ratings of 
whether the prggrom^met the fellows' educational needs correlated with all of the items 
from the other' sources In addition, faculty judgments of. whether the fellows wouft become 
better teachers correlated with fellows' judgmenf^ of effectiveness and satisfaction but 
noKwith evaluators"^' judgments. Faculty members' own' reactions to the program and their 
rqtTuga*^ its value to the fellows did not correldte significantly wjith any of the rotiiigr * " 
of satisfaction and effectiveness that fellows or evaluators made; 

Table JO. Correlations among Measures of Effectiveness apd Satisfaction 



Source • 


Item 


Faculty • ' 
16 18 20a .26 


Fellows ^ 
23 *24»'. 31 


19^ 


Faculty 


16. Reaction to ExTFP 

\j8. D.idW^eilows become betVer tedcher? 

20a. ExTFP valuable for fellows? 

26. ExTFP meet fello^^ needs? 


-- '.43 .^3 '<69^ 
.43 -- .43 ,.62 
.63,. 43.,-- .49 
.69 .62 .49 ~ 


..^ .31' .34 
.43 .46 .55 
.02 .01 .47 

.41 -.5? .61 


s24^ 
.14 
,12 
.44 


Fellows 


■ ' \ ^ 

23. ExTFP meet your major need? 

24. Reaction to ExTFP • ^ 

31 , EixTFP^help yrour teaching? 


^76 .43 .02 .'.41 
.31 .46, .01 .58 . 
.34 . 55 .17 a61 


■— .%1 ./4' 
.61 -..^ 
.74 .81 ~ 


.40 

..60 
.65 . 


EvaluotOf 


■ ■ ■ * # ■ ■ ' 
19. ExTFP meet fellows' needs? 


.24 .14 .12 -.44 
^ ' 


.40 :60 .65 





file -fact that thq fellows' judgments of ef/ectiveness correlated with those by 
the evaluators and also with faculty members! es/imates of whether the ExTFP f*et the 
f-el lows' educational ne^ds is encouraging evidence of consistent, reliably-ascertained differences 
between prbgram% in their effect^ehess . Wt^y the otHSp^*faeul>y measures of satisfaction 
anS effectiveness did not also coiwlate j^jtl^he items from tih^feUows' and^he evaluotor 
questionnaires is not immtediately clear. * , * ^ 



-33- 



49 



D. The Relationship Between ^rbg ram Effectiveness and Program Structure * 

Under tfns heading'we will consider the correlations between program. effecti>^eness 
and three classes of variables: (1) achievement of, the e^n bloc approach, solidarity arid 
morale, "(2) relations among departments and institutions, arid (3) the^role of the director. 
In each case, we first present the correlation among variables within the set, then thefr 
correlatiftrfs with' effectiveness. ' - ' 




The En^Bloc A]!)proach, Solidarity, and Morale^ 
%. \ ^ S 

^ a Correlation dmong meosures. We have already learned (a) that a genera IJy 
hfgh level of solidarity was achieved in all of the programs. but (b) institutions wad 
a consi^tentlj' higher fevel of morale than did others. Table 11^ presents the correlations 
a mono the dtff efent Ja[fi|gs of solidarity and rnorafe . AJI but one of tbd ,lC|^orrelation 
coefficients achieyiSfl Hf istical^ sic)nif icance, the m&n correlatiort being ISQ. There-, 
was, tften, remarkable agreement between gourdes as to. whicpf progranv wei^e characterized 
'by a very high degree of solidority and niorale and which were not* It should be 
noted tha^the lowest correlation^ this^table,^ .30, was ^etween the evafudtors' ranting 
'of Successful achievement of th ^yi blo c approach and the facujty rating of sfbd^Qniiorale , 



Table 11 . Correlations among Measures of ^Solidarity and j^orale 



Source^ 


• Item 


Faculty , 
^ ' 48 49 . 


47 '49 


Ev. 
17 


Faculty 


9 ^ . , 

48* Student solidarity ^ 
49/ Student morale 1 


. .71. -r 


-.57 .68 
.57 ^67 ' 


>- 

.45 
.30 ' 


Fellows 


47^ Student solidarity* , ' 
49: Owri morale 

^ ^ — . 


• .57 .57 
. .68 .67 :, 


- .76; 
• .76 - 

1 . 


• ..49- 
.56" 


Evqjuator 


IZ^En brec successful 


.45 '.30 
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b^. Relatipn of scffidarity^an3 morale to satisfaction and effectiverwss. •Table' 12 
pre^nfs the correjations between measures of solidarity ord morqMe and those <5f satisfaction 
and effectiveness.^ It is' evident that-+he majority of these correlations were-quite high,* 



even whe, 
evalubtOTTatings 
effectiveness .» 




ings/w^re obtained froqi different sources, except that fellow and 
lidarity did not correlate .significpntly with fqculty measures of 
erage correlation <5f faculty ratings of solidarity with faculty 
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ratings of effectiveness was4 .50, with fellows' rotings'of effectiveness, + ,46, jond witli / 
the evoluator rating of effectiveness, + .47. The overage correlation of;fel lows' ratings of 
solidarity with their own ratings of effectiveness was4 .47,"V/ith the evoluator' to ting of 
effectiveness, 4. 42, Kind with foojlty ratings of effectiveness,, 4. 2^. .Finally, the evoluator 
roting^of achievement of the en bloc -opprooch ccM^loted 4 .71 wij^h the evoluator measure 
of effectiveness, hod on overage correlation o^4^1 with fellows' measures of effectiveness,' 
and an average correlation of only 4. 18 with fo^culty rati>ig$f pf-^ectiveness. Whatever 
the cajlfil factors that may.be involved in these correlations, it is jcleor thot thje achievement _ 
of the en bloc approach^ and Of solidarity ofKl morale was ossocipted witK program effectiveness^ 
especially as viewed by fellows and evaluatorS.. f - 

ft 
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'pbleJ2, Correlations of Solidarj^y and Morale with Program Effectiveness 



m 

Source 


r ; 

Item 4I . i» 


Faculty, 

W 18 20a 26 . 


Fellows 

23 24 31 


Ev. 
T9 


Faculty 


48;' Student%ii<kirity 
49. .StiJdent morale 


.43 '.42 -.25 -.44 
.62 .62 .62. .63 ' 


. '.35 .60 .58 
,.22, ,47 .51 


.57 
.37 


Fellpw^ ' 


MZ, Student solidarity 
'49. Own moral#* > 


• 27 .37' .2Zm;^2- 
.30 . 26 . 22 ^30 


.13- .57 . 41 


'.44 . 

;40' ; ' 


Evaluotor 


17|p£nJ»|oc suctessful 


.28 .08 .J3 .21 


.^^^1 .44 


".71' ; 




2. Relations o^opg Departments and Institution^ .' 1 * 

• ' ^ a . Correlations cfmong measures. Under this h^odTnywill be considered ^vojuotpr . 
^ and fallow ratings of^cooperation'^belweert^depairtments, and evJIiJblhor judgments - ^ 

of whether the-institutipn had established reJationshrp^ with the local scho^ systems . ^ * ' 
J It will be 'remembered that respondents reported great variability djnpng program/ in 
• the' extent of cooperbtion between subject-matter and teachnpr-^ducdtion departments, ^ 

ond in the amount of cooperation with loco I' school sysf^ms* Table 13 presents the * 
' correlations among the three, measures of 'cooperation;. None of 'these qorl-e lotions is 

higher th«^4 ,2] . This independence of on'p set^irf responses from another reveals, fifst, 
<^hat accoi^ingJaevpJua|pr$^ reports, wf^ethe^bject-mbtter and teacher-education 

d epartme 11% <S^^e rated 4Tad no bearing upon whether codperation was estobfished between ' 

the institution and local school s/stems. Second, the low qorreftfionTijeans that 
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^judgments of departmental cooperotj^Riyp by fellows were ^itheriixasecJ upon different^ 
criteria from^<>se used by evaluato^B^hdl the two groups used their'crlteria differently 
\n these judgmeiW* ^ 



< * 

Table 13. Corre'lations among Measures of Departmental dnd Institutional Cooperatlorj^ ; 



1 

Souroe 


' ~ 1 

/item / . 


Fellows 
•38 ' ' 


Evaiuator 

3 4 ■ 


-ellows 


38. Departments cooperate , * 


-c^ 


.15 .2V 


Evoluotpr 


. 3.. Departments cooperate 
' 4, Coop, with locpl schools 


.15 
'21 


.18 

.18 

i 



b. .Correlations of m easures of cooperation with program effectiveness. 
Despite the lack of correlation among, these meOsures of cooperation, we see in Table 14 
>(TOt both F'ellowi' and ^valuofors* assessments of dep<5rtmerital cooperation were correlated 
slgmfitqntly-^vith ratings of effectiven^made by fellows and eyaluators; however, , 
they were co'nsistently urtSorrelafed with faculty ratings of program effectiveness. Thus, 
althoughjheJellows may Rave us^d cUfferent criteria from evaluators in judgiM departmer^a 
cooperation, by either criterion, programs that were rated as haying a relatively high 
degree of cooperation among departments were more Jikely than not fo be adjudged 
eff^ffi^e . The degree of cooperation with lg>cal school systems, as reported fcy evaluators, 
was substantially unrelated to any measures of effgdtive/iess. ^| 
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Table "14. * Correlations of Departmental Cooperation with Program Effectiveness 







1 ■ ji 


'■ Faculty 






Fellows 




.Ev. 




Source 


Item > ' • 


' T6 .18, 


20^ 


26 


. 23 24 


31- 


19 




Felloes 


38. Departments cooperate 

: vi- 


.12 ..21 


-.11 


.35 


.-56 .70 


.60 


.54 








-.04 -^20 




—r- 










Evaiuator 


3 . Departmerrts cooperate 


-.1-2 ' 


.06 


.40 .32 


.47 


.-44 




e 


4. Coop, with loqal schools^ 

• 


-.10 -.f) 


-.11 


.27 


.14 .26- 


.31 


.35 




E. The Relationship Between progrom Effectiveness and Program Strotegy ' ^ 

* • " lb 

' Th©.reader will recqll that the general topic of program strategy subsumed ratings 
of the utilization of fello*«vs' background) the arnount of'^competitiveness and the work load, 
the Involvement of the faculty, and th| extent of innovation in the programs. The 
corralation of each of these varipSles jwith effectiveness will be discussed In turn. 

Utilization of Fellows' Background 



prgoni 
were 



\ ci C A-reldtlon among measures. In their judgments of whether the program's y 
izoffon took into account the extensive experience of fetlows, evaJua+ion teams 
dislintt from the other jhree groups in saying that the majority did not. Though . ^ 
they disagnfeed with the fellas in the extent to which they felt tlwt fellows' backgrounds 
were uHmed, the evaluation t^ams clearly arrayedjnstitutiohs on this variable in about 
the same order as did the fellows, for the congelation between ratings from these two 
sources was+.45. This correlation, in fact, was the largest correlation In the matrix 
(Table 15); the only other significant correlation is that between fellows' judgments on this 
variable and faculty statements that the program was modified to take advantage of 
fellows* experience. ^ , * 



Table 15. Correlations among Measures of Whether the Programs Utilized Fellows' , ^ 
Backgrounds, r ' / * 



Source 


' , '5^ 
Item f 


. Fatftulty. 
45 , 46 


Fellows 
42 


Ev. 
16 


Fxiculty 


45. , Effort to use i$D<perience . , ^ 

46. Modif)^ prog, for exp^ienc^^ 


.06. 


- .05 - 
^ ' .38 


T22 
.07 


Fellows 


-\ ' f 

42. Build on experi«nce " ' . 


v.- ^ 




.43 


• 

Evaluator 


''ft ' * 
16. Take acctXAi experience ^ , 


.22 „ .07 


,45' ^ 
-1^ - 





bi Correl ations of utljizotjon of participants' backgrounds with progrant 
effectiveness and soTidarit/ i ^ As \l st»Wn in Table 16, foculty judgments of whether the 
program utilfzed feilowsii experiences were not significantly related to any measure of 
effectiver^ss; faculty staterpepts that* the program was rpodified to take advantage of'the 
fellows' experience were gen^r•ally relcf#ed to thjp faculty's owa estimates of effectiveness. 



but not to those by fellows and by evaluators. On the other hand, evoluafors' and* fellows* 

judgmentvon this.same measure were significantly related to every feHow and evaluator 

measure of effectiveness and also to faculty judgments that the program met the educational ' 

needs of the fellows. We may conclude, then', that the programs whose organization built 

best on the backgrounds of the fe-llows according to the interpretation of the evaluators 

ai|||^e fallows (and we do nrt^know the criteria on which these sources based their interpretations) 

were also adjudged to be9nore effective^ ' ^ 



Toble 16.^ Correlations of Utilization trf ParticfpoRts* Backgrounds and Program Effectiveness 



Source 


Item 


Faculty 

16 20a • 26 18 


Fellows 

23 . 24 31 


Ev. 
19 


« 

Faculty 


45. Effort to use exp . 

46 . Modify prog / for exp . 


-.19 . .04 --.13 '-..^S' 
.54 .31 .53 .27, 


-.2^ -.06 -.10 
.12 ..21 .23 


.06' 
.05 


Feflows 


42. Build on experience 


.27 . .16 .51 .53 


.48 .71 .76 


.47 


Evaluotor 


16. Take- acct . 'of exp . 


.16 . 04 . 44 . 03 


.34 .60 ,60. 


.71 



2 . Com p etltiverll^ an(J Work Lbod « , ' 

. ^ *• . 

, a . Correlations among measures > The (^relations reported in Table 17 lend - 
"empirical support to our earlier conclusion that,*th^iaculty judgments about student work 
* lood/epresented d positive statement about the fetlows, rwt an objective assessment of 
t^ie amt>unt*of wor were required to do; ev$ry correlation of the faculty judgments 

*on this item with those of fellows or evaluators was negative. On t^Te other hand, fellows' 
judgments of their work load correlated positively ond significantly with those ^of the 
evaluators. It should be noted, in add^tionT^hat neither of these last two measures — 
follows' and ^valuators* estima^ of the work load — .correlated significantly with fellows' 
statements abo'^ the level of competitiveness in their programs; clearly, ^"el lows could 
believe they were overworked in either a coqj^e^itrve or a ruih-competitive atmosphere. 
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Table 17. Correlbtibfs among Measures of Competitiveness and Work j.odd 


J 


* * 




Source 


~ rf ^ 

Item 


Faculty 
• 35 


'Fellows 
26 ^ 


29 

* 


Ev. ' 
14 ' 






Faculty 


35i^ Did s,tudents work hard 
_jt 




-.39 


.31, 


• -.26 


ri 


w 


Fellows 


V 

26. 'Work.Lood _ . 

29. Level of competifiedr^ 


, r.39 
-.31 


'V 
.28 


.28 


.54 
.01 


• 


v> 

i 


Evaluotor 


14. Work Load 


-.26 . 


.54 

' s 


.01 








b. 


Re+Qtionship of competitiveness and work loa 


d to prograrr 


1 effectiveness 





actually represent favorable judgments of their performance is given ?V) tow 1 of Table 18. 
This measure correlated positively and significantly with every faculty rating of program 
effectiveness and with two of the three effectiveness ratings made by fellows. 

' * A different oattern field for fellows', ratings of their work load. These judgments 
showed a high negatlive correlation with fellows' opinions thai the program was stimulating 
and interesting, and moderate negative correlations with fhe other ratir>gs of program 
effectiveness by theVellows and evaluotors as well as with faculty jpdgmients of whether 
the program producedN^etter teachers, .tvaluator ratings of wofk load fiso showed negative 
correlations with fellow and evaluator ratings of effectiveness. As to the level of 
competitiveness in the program, while the correlations of this measure with judgments of 
effectiveness and'satisfaction Were consistently negative, they barely achieved statistical 
significance in on[y two cases. In short, programs in whiqh fellows and evaluotors reported 
that the v<ork lood was excessive tended also to be programs which received low marks, ^ 
for" effectiveness and satisfaction, but'CT program that v^s viewed as competitive was not 
necessarily ineffective^. ' '"^ 

Table 18^. Correlatjons of Competitiveness andWork Load with Program Effectiveness ' 



Source 


Itenf 


Faculty * 
U 20a 26 18 


Fel lows 

23 24' 3J.. 


' Bv. 


Faculty 


35. Wprk hard? 


.49 -.41 .63 .62 \ 


^ .22 .51 .46 


.24 


Fellqws 


26. Fellows' work load * 
29. . Competitiveness 


-.34 .0;j -.21- - .43 
-.17 -.28 -.24 -.39 


-.35. -.72 -.30 
.01 -.39 -.30 


-.02 
.04 


^luator 


14. Fellows' work loa^l ^ 


-.26 . 08 -.30 - .26 ' 


-.g4 -.44 -.35 


-.38 
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3. In volvement of the Faculty 

' o. Correlations among measures. There were five it«ms Hiat bore on th^ faculty^ 

involvement in the ExTFP: two of^hese asked^he faculty v^hether tH* €xTFP had been ^ \ 
challenging and satisfying, one a$ked' evaluators if the faculty had been challenged onciy**^ 
stimulated by the ExTFP, and two asked fellows whether the faculty had been accessible • 
&nd helpf ul. The correlations amonj ftiese measures, presented in Table 19 show that 
almost the only significant correlations are between measures fi;om the same questionnaire. 
Thus, institutes in which the faculty said they were challen^d were also those in'which 
the faculty found the teaching satisfying; schools in which fallows reported the facdity were 
accessible* were schools in which fellows said th^ faculty were helpful. The only siginificant 
correlation between i>ems from different questionnaires was between fellows' reports of 
faculty helpfulness and evaluator ratings of fdculty stimulation. 



Table 19. Correlations among Measures' of Facj^lty Involvement 



Source 


T""^^ • 

Item 1 


Faculty 
^74 25 


Fellows 
41a 41b 


Ev. 
9aN*^ 


Faculty 


y m i ' — 

^4. Was teJs^ching. challenging ? 
25. Was teaching satisfying? 


.56 

.56 


.19 .30 

.10 ,28 ' 


.33 
.14' 


-ellpws 


41d. Were-faculty accessible ? 
.41b. Were faculty helpful? 


.19 .10 
.30 .28 


.87 

.87 ^ 


.24 
.41 


Evaiiic^or 


'9a. Was staff challenged? 


— TT"* — *" 

.33 .14 


.24 ■ .41 

J*— J 





b.' Correlations bjgtween faculty involvement aqd effectiveness. The pattern 
of. correlations between fdculty involvement and effectiveness, presented in Table 20, 
is not easy to understand. One of the faculty measures, statements about whether the teaching 
experience was satisfying, correlated with virtually every measul^ of effectTveness-and 
satisfaction i^^hops because it might, itself, be called a measure of satisfaction, 
Evaluaiors' judgments of whether the faculty.was challenged were correlated with fellow ^ 
and evaluator, but not faculty, meosures of effectiveness: Fellow ratings of the faculty's 
helpfulness and accessibility correlated signilficaptly only with their own judgments of 
effectiveness. 



Table 20. Correlations of Faculty Involvemfent' wtth Program Effect! ^enetss 



• 

Source 


Item s 


^ Measures of Trogram Effect ivenesS" 


■ Faculty 
16 20o 26 1& 


"fellows 
.23~ 24 31* 


Ev. 
T9 


, 7 

Faculty 


24. Teaching choll . ^ 

25. Teaching %atisf . 


^42 .09 . 31 .28 V 
.75 .27 .55 .46 


.26 .25 .26 
.31 .45 . 50 • 


.18 

.30 


Pel lows 


— — ~7 •. 

41a.' Fac. accessible 
41b. Fac; h^ful 


•^'.01 -.25 -.0-1 .23 
.14 -.11 .38 


.58 .40 .29 
.65 -.66 .49 


1 

.15 
.32 


EvaluQtor 
4 


1 

9a, Fac, challenged 


.16 .01 .24 .08 


.44 .51 .45 


.68 



4 . Innovativeness 



ks have seen, on the two measures of innovativeness, neither faculty members 
nor e^aluotors reported any appreciable degree of innovation, l^lor did measures of 
innovation from the two sources var^ jointly: the correlatioi% between them was JO. 

On the other Kond, as may be seen in Table 2l, the faculty esfimate of innovativeness 
•was related to every measure of progrq^p effectiveness; the average correlation of this variable 
with faculty ratings of effectiveness wpi .55, wit>i fellow ratings of effectiveness, ,.48, and 
with the evqiuator rating of effect! v^ess, .38 (Tpble 21), Evaluotor ratings of innovativeness, 
on the other hand, were. significantly related pniy to the evaluator measure of effectiveness 
and to faculty Mjdgrpents of whether the fellows became better teachers as a reM:4i of their 
experience. We see, then, that by the evaluators' standards of innovation, our earlier 
generolization Kbids up: prograr^^^uld be effective whether they were extensive^y innovative 
or substantiori)^ traditional . \\ should be noted that>fhis question asked faculty members 
whether they bad observed innovative teaching methods or practices; in view of this wording, 
their judgments may have* reflected inventiveness in some one teacher's performance rather 
than .innovativeness in the over-all program. If so, the meanir>g of this variable's correlations 
with effectiveness is considerably different from the meaning that would be carried by a 
correlation with innovo^tiveness in the progrcim itself. 

An alternative explanation of the correlation between these facul^ ratings and 
effectiveness is that the current popular emphasis on innovation in education has served 
to make "quality" and "innovation" in some respects synonomous^for many people. Thus 
rtxjny faculty members may haye felttbat if they j^udged their program to.be successful; 
it Biust have been innovative, as welk 
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«Table 21 . Correlations of Innovativfiness with Pr'ogranf EffectivJRess 



• 

Source 




Measures of Effectiveness 


Faculty 
■ 16 20a 26 18 


Fellows 
23 ^ 24 31 


Ev. 
19 


Faculty 
* 


2/. 1 n nova tf yen ess , 


.59 .41 .79 .48 


.50 '.42 .53 


; 

.38 


Evaiuofor 


^ 2 . Innovativeness 


.11 ,18 .01 ,46 


-.14 .03 — 


.44 



-^5* Effects on institutional dev elopm ent 

♦ 

a, 'Correlations among meosures . We hove seen above-tt)dt the three^sources 
differed remarkably in their estimates of the Program^ effect on mstitutiorKil development. 
As mi^ht be expected, the pattern^of correlatidhs between var idb I es showed the same lack 
of correspondence (Table .22): estimates by faculty members of the Program's value to the 
institution were uncorrefated^with evaluators '^ratings of both effects on tetacher training and 
effects on departmental development. At the same time, the high correlation between the 
two evaluators* judgmi^nts shows that institutions at wfiich evaluators felt changes were" 
made in methods of 'teacher education also ^ere judged tote institutions where departmental 
development was affected. This relationship, actuajly is partly determined by the fact that 
the "department" whose development was b*eing affected; thai is, the hom^ department of 
the program, was in half the cases the d^partmefft of education. 



Table 22. Cprrelations among Mea^sures of Effects on Institutional Development 



Source 


Item 


Faculty 
20c 


EvaFuator 
1 8 


"acuity 

• 


20c. Value to institution 


- / 


.16 .07 


ivaluator 


1 . Effects on teachej training * 

8. Effects oit^epartment, jdevelopmeQt 


.16 
.07 


.56 , 

.56 -- 



b.^ Correlations between institutioTioi development and effectiveness. Table 23 
presents the correlatrons. between measures of institutionoj develqpment and judgments of 
satisfaction and effectiveness. Globol judgments by faculty members of the Program's - 
value to the institution correlated positively with their. own ratings of satisfaction and of 
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program effecfiveness, buf were onl/ minimally related to such rafinl^by t)fher sources. 
Evaluafoc judgments of fhe l^rogram's effects upon the hos^ insfifufion showed low positive 
relafionshtps fo fheir own and fellows' ratings of effecfivehess,, buf were unrelated to such 
ratings by faci/lfy members. In shorf, if appears fhaf a program cogld be effective in ■ * 
fratning sfudenfs wifhouf, necessarily, serving as fhe impetus for extensive changes m 
the pattern or organization at the host insitituion. ' # * " 



c 



r 

Table 23. Correlations of Effects on Institutional Development with Program Effectiveness 







Measures of Prograjit Effectiveness 


: 

Source 


« 

Item 


Faculty 

16 20a 26 1& 


Fellows 

23 • 24 31 


Ev. 
19 


"acuity 


^Oc. Value to insti. 


.70 .47 .47 .63 


.07 .41 .35 


.29 


Evaluator 


•'1 / Effects on teacher tr. 
8. Effects On dept. dlv . 


m 

.04 .10 MO .20 
.14 .03 .07 .22 • 


.29 .44 .37 
.14 .27 .31 


.35 
.35 



E. Summary: Correlates of Effectiven^s \ • ' j 

' ^ 

* Detailed comments should be made about two "aspects of thes^ results. The first 
has to do with the reliobility of judgments, with whether two judgments which seem, on 
^J^j surface, to be oskingVhe some'question^do, in foct, correlate with one another; the 
second is the consistent correlotes of satisfoction ond effectiveness. 

1 . The rel iability of the judgments 

In generol, when o single source was asked more than one question on the same 
topic or logicolly reloted ones, the responses to those qi;e$tior)s were positively correlated, 
f Thds, foculty, fellow, or evaluator responses to one item showed generally high correlations 
with responses by the some source to other items whose content was simitor . For example, 
institutions'whose faculty* members said the Progrom was stimulating ard interesting throughout 
were ciso the Ovies whose faculty said that the Program was of great value to the fellows, 
that itihelped the fellows become better teachers, and that it met fellow^s* educational needs. 

HoweN^er^ responses by different sources to items that were similor in content did 
not always conrelate significantly. Two sets of items — satisfaction and eff^ctiverfess, ond 
solidarity, ond morale showed marked consistency across all three sources; for each set of 
items, the ratings that institutions received from faculty members paralleled those given by 
fellows and also, to a lesser exteht, those given by evaluation teams. On a number of 
other factors, responses by faculty members were substantially uncorrelqted ^ith those of 
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fellowsond of fevaludtors. However, the responses of fellows ard those of evaluation 
. teams correlated virtually aoross the board — in ratings^of satisfaction and effectiveness, of 
solidarity and morale, of fellow^' work load, of whether *the feHows' experience was utilized, 
and of the exJent of faculty involvement in the Program, responses from these two sources 
correloted significantly; only in*judgment5 of wKe^ther there was cooperation between different 
departments were fellows' judgments arnl those by evaluators uncorrected. For qpe set of 
items — estimates of fellows' work load — faculty judgment were inversely related to those 
^ from the other- twp sources, probably because the question that was aslced of faculty members 
evoked judgments of the level of fellows' motivation instead of objective assessments of 
tfieir work load. In short, there was consistently high agreement between judgments 
by fellows arxJ those by evaluation teams; agreement between thpse two sources and the 
faculty was largely restricted to fwo area^: (a) satisfaction^pg effectiveness and (b) solidarity ^* 
and morale. / 

^ * Itjs interesting to speculate about the meaning of this differential pattern of 

correlations. Since the three sources viewed the program from different perspectives, the 

{ pattern of correlations probably^eflects such differences. It might be argued, first of all, 
thaf whether a program was very effective orvelatively ineffective, and whether its ^ 
- fellows had very high or r^latl^ely low morqie could be determined by faculty and feflows 
alike from evidenc^ that is public and common. People discuss with one another how much 
they have learned and hov/ well it was presented; there are indicators of group solidarity and 
morale' which pimost any adult can see and identify. On the other hand, more subjective 
crfteria Ofe called intoplay for judging whether and how much the fellows' prior experience 
was utilized by the program^or how deeply the faculty was involved in the program. 
Such questions are probabl|J^ess frequently discussed, the bases for decision about them ^ 
lejs commonly shared, than the topics of effectiveness or group soiidarity. If this is true, \^ 
judgments on these latter topics would be more likely to reflect the biasing effect of the judge's ^ 
social position. ^To be more^ecific, it seerps likely that the faculty's institutional position 
made it unlikely that they would learamuch about the fellows' past experience and its 
relevance to the course material, or about fellows' judgments of whether the faculty was ^ 
involved in the program- In the. absence of explicit information, fhe faculty was doubtless ^ 
likely to respottd in a manner coWistent with their desire that their own progr^fm be rated 
effective and "good . " v v '"^ 

^ These considerations would account for the lack of correlation between responses 
of fellows and faculty on issu^.of this second type,' but not necessarily for the fact that 
jpdgfnents of evaluators paralleled ,those of the felbws instead of the faculty when the 
latter. sources disagreed; Perhaps their discussions with the fellows exposed evaluation teams 
to information that was n9i^vai^able to the faculty; alternatively, perhaps evaluators 
considered the faculty/fo be more personally involved than fellows in the outcome of the 
evaluation, hence moK ejlW y to betiased in their judgments. * ^ % 
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-Whether One of these explanations or some other one can account for the results, 
oLcourse, requires information that is not available in the present study. - Concerning the ^ 
reliability of Judgments, we have seen that judgments' made by a single soured on a single 
issue were quite reliable," ttiat cor»istent positive^ correlations were found between* the. three 
different sources in their ^judgments of effectiveness and of morale, and that evaluatfon teams 
and fellows consistently agreiB with each other, but not with the faculty, 6n other Issues. ^ 

2. Correlates of satisfqction and effectiveness ^ 

• 

Programs that ranked high on satisfaction amf rated effectiveness (a) were adjudged 
by faculty and by fellows to have a high degree of solijdarity and morale, and by evaluation - 
teams to have been successful in achievinig esprit de corps through tMe en bloc approach; 
(b) were successful in the view of fellows and of evaluators in utilizing the previous experience' 
of the fellows; and (c) were adjudged by fellows and evaluators not to have required arf^ 
altogether unrealistic bmount of work. ^ 

The fact thj^t effectiveness and morcfle went together is not surprising; it reaffirms 
a long standing^-clommon-sense generalization. It is interesting to note, however, that 
effectiveness wo^ correlated only with judgments by fellows arKi evoluators of whether the 
fellows' backgrounds were utilized and of work, load; judgments by faculty members on the 
last two issues did not correlate with the same judgments by the other two sources. This 
suggests that the facultyond directors may often have been uninformed of the fallows' 
attitudes cm these and other issues; indee'37 spontaneous comments by evaluation teams 
suggested^that such was often the case. This, in turn, has implicotions for the conduct of 
programs in institutions where the fellows thought that their work load was much too heavy, 
or that they were to5 seldom able to contribute from their own IcnowJedge and experience 
to the educational program, and ^here the faculty-and director were unawore of these attitudes. 
It seems likely that infon!|ption about fellows' attitudes might have induced the staff of 
the programs either to change some part of their educational structure ond content so as 
to meet the fellows' objections, or to clarify fpr fellows and staff alike the reasons for 
retaining an existing system. These actions, in turn, would likely have made such programs 
more enjoyable and effective. The obvious suggest*ion, then, is that some programs 
might hove been much more effective if the fellows' views on sensitive issues had been more 
effectively coipmunicated to t^se faculty and the director. Clearly, the primary responsibility 
for ensuring that such communication takes place rests Nyith the director and his staff, 'not 
with the fellows. , ^ 

Although fellows and evaluators ,did. not ogree as to which programs had o great 
deal of cooperation among departments and which dfid not, by either the evaluators' or 
the fellows' criterion, programs. vyith such cooperation were more- effective than those with<^t . 
it. Similarly, although faculty members and evalOotbrs did tet ogree as to which host 
institutions benefitted most from^the Program, those institutions that either group judged- 
to have benefitted most were rated as most effective. Two other kinds of^ questions shQwed 
inconsistent patterns oi relationships with satisfaction and effectiven^/ Ratings by eaph 
'source of the^xtent of faculty Involvement in the program were corrl^ted with effectiveness^ 
fis rated by that source but not as rated by the other sources. - 



-45- 

52 



> 



• •• . / 

FinaiiyV Hiere was a significant correlation between faculty ratings of innovativene^- 
and every measure of effectiveness — institutions whose faculties were impressed with 
the innovations'thot had been introduced throujgh the Program were consijlered to be relatively 
effective by foodlty, fellows, and evaJuators alike; however, evaluator ratings of . • 
innovatlveness were only marginally correlated with ffffectiveness as judged by evaluators 
and were uncorreJated with sudh. judgments by the other two sources. We have suggested 
that faculty ratings of innovativeness may hove i;Bf lected theij* own involvement in the program 
more than objective judgments of this phenomenon.^ 



We have afready npnarked'? that' one should, not infer causation from correlation. 
Hopefully/ future studies in this series will help further to clarify the factors that account for 
differences among institutions. Our caveat against confusing correlatiort'with caOsation, 
however, does not appl^ to the relationshij^betweWn the director's behavior and program 
effectiveness, ,The experience of the evaluation teams strongly suggested -^that an, energetic, 
persuosiye director with institutional power commensurate to his responsiblJities could play 
a major role in assuring the effectiveness^of the program. Conversel^t, a prorpising program 
wassometimes rendered less effective by an inept director, one with insufficient time to give 

duties/ or one denied the power to institute and carry though both general policies 
and the specific procedures necessary to implement those policies. 
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